What’s New

PyTorch

AI

Tensor and neural-network library for Python and C++, with GPU acceleration and eager execution.

Latest v2.13.0 · by PyTorch FoundationWebsitepytorch/pytorch

Changelog

v2.13.0

PyTorch 2.13.0 Release

PyTorch 2.13.0 Release Notes

Highlights

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Tracked Regressions

ROCm wheels break torch.compile on CPU in environments without a GPU

Running a torch==2.13.0+rocm7.2 wheel in an environment where no GPU is available (torch.cuda.is_available() is False) breaks torch.compile on the CPU path: the first compile raises RuntimeError: Can't detect vectorized ISA for CPU (#189194). This is a regression from torch==2.12.1+rocm7.2, which compiles CPU code fine (detecting e.g. VecAVX2) in the same setup. The 2.13 ROCm wheel appears to rely on something present in the ROCm builder image to detect the CPU vectorized ISA, so it works when run on a ROCm image but fails on a plain CPU-only image.

Workaround: run the +rocm wheel on a ROCm image, or install a standard CPU/CUDA build for GPU-less environments.

Backwards Incompatible Changes

  • Stop building CPython 3.13t (free-threaded) binaries (#182951)

    Upstream pypa/manylinux removed CPython 3.13t (free-threaded) on 2026-05-07, because 3.13t was experimental and has been superseded by the now-non-experimental CPython 3.14t. As a result, PyTorch 2.13 no longer ships cp313t wheels (Linux, Triton, and related artifacts). Users on the free-threaded interpreter should move to Python 3.14t.

    PyTorch 2.12:

    # cp313t (free-threaded 3.13) wheels were available
python3.13t -m pip install torch

    PyTorch 2.13:

    # Use free-threaded Python 3.14t instead
python3.14t -m pip install torch

  • Bare PyObject is no longer allowed in operator schemas (#184209)

    Bare PyObject was accidentally accepted in operator schema strings in PyTorch 2.12. This was undocumented and is now rejected, since torch.compile does not support arbitrary PyObject inputs to custom ops. If you parse or register a schema with a bare PyObject argument or return type, you will now get a schema parse error.

    PyTorch 2.12:

    >>> from torch._C import parse_schema
>>> parse_schema("foo(PyObject x) -> ()")  # accepted

    PyTorch 2.13:

    >>> from torch._C import parse_schema
>>> parse_schema("foo(PyObject x) -> ()")  # raises a schema parse error

  • Remove Bazel build support (#180883)

    The Bazel build was never broadly adopted and still depended on the antiquated Bazel 6, while the wider ecosystem has since moved to Bazel 9. All Bazel build files and CI jobs have been removed. Users building PyTorch with Bazel should migrate to the supported CMake/pip install build flow.

    PyTorch 2.12:

    # Build PyTorch with Bazel
bazel build //:torch

    PyTorch 2.13:

    # Bazel build files have been removed; build from source with pip instead
pip install --no-build-isolation -e .

  • StorageImpl's built-in copy-on-write (COW) materialization is replaced by a pluggable materializer hook (#179063)

    StorageImpl no longer knows about COW directly. Its internal COW entry points StorageImpl::is_cow(), StorageImpl::maybe_materialize_cow(), and the friend cow::materialize_cow_storage() have been removed in favor of a single pluggable MaterializeFn hook (void(*)(StorageImpl*)) that a backend registers to run once, on the first mutable data-pointer access. COW is now just one consumer of this hook (c10::impl::cow::materialize_cow), and all COW behavior (lazy clone, refcounted shared data, copy-on-write) is unchanged. This also gives accelerator backends and eager-mode graph compilers a zero-fast-path-cost place to commit deferred allocations or materialize symbolic buffers on first mutation.

    This is a C++-only change. It affects out-of-tree backends/extensions that called the removed StorageImpl COW symbols directly; they will fail to compile against 2.13 with errors such as no member named 'is_cow' in 'c10::StorageImpl'. Migrate to the new hook API (set_materializer() / has_materializer() / clear_materializer()).

    PyTorch 2.12:

    // Detect a COW storage and force it to materialize.
if (storage.is_cow()) {
  storage.maybe_materialize_cow();
}

    PyTorch 2.13:

    // Register a one-shot materializer; it runs on the next mutable-data access
// and then clears itself. COW registers c10::impl::cow::materialize_cow this way.
storage.set_materializer(&my_backend_materialize);  // void(StorageImpl*)

// `has_materializer()` replaces `is_cow()` for "is a deferred materialization pending?"
if (storage.has_materializer()) { /* ... */ }

  • Convert shared_ptr<Node> to intrusive_ptr<Node> (#181139). This changes the signature of Tensor::grad_fn. Accesses to Tensor.grad_fn() should change from std::shared_ptr<Node> to c10::intrusive_ptr<Node>. Similarly, construction of a C++ autograd function should change:

    PyTorch 2.12:

    std::shared_ptr<CustomCppNode> node(new CustomCppNode(), torch::autograd::deleteNode);

    PyTorch 2.13:

    auto node = c10::make_intrusive<CustomCppNode>();

  • The minimum supported NCCL version when building from source is now 2.23 (#186292)

    PyTorch now requires NCCL >= 2.23 at compile time, and the preprocessor/runtime gates that guarded NCCL features introduced in 2.23 or earlier have been removed. Users who build PyTorch from source against a system NCCL older than 2.23 will hit compile errors against the dropped gates. Upgrade the NCCL installation to >= 2.23 to build. The prebuilt PyTorch wheels already bundle a compatible NCCL, so pip/conda users are unaffected.

  • Remove named tensors (#173895)

    The named tensor feature (a long-deprecated prototype) has been fully removed to reduce overhead and code bloat. All associated Python and C++ APIs are gone, including Tensor.names, Tensor.rename(), Tensor.refine_names(), Tensor.align_to(), Tensor.align_as(), torch.align_tensors(), the names= keyword on factory functions (e.g. torch.zeros, torch.empty, torch.ones), and the C++ Dimname / DimnameList APIs. Code that previously relied on named dimensions must track dimension order positionally and avoid usage of any of these now-removed APIs or op overloads.

  • The onednn::qconv2d_pointwise.binary and .binary_tensor operators no longer alias their input but rather return fresh tensors. Previously these ops mutated the qaccum input buffer and returned it directly, violating the PyTorch invariant that custom operator outputs must not alias inputs. This silently bypassed aliasing checks via the old -> Tensor(a!) schema and would become a hard error in a future PyTorch version (as mentioned in #182063), so the schema and implementation were corrected to return a fresh output. Most users are unaffected, only code that calls these ops directly and relies on the in-place mutation of qaccum must now read the returned tensor instead. (#177171)

Deprecations

  • Custom operators that return an output aliasing one of their inputs are deprecated (#182063)

    When a custom operator returns an output that is the same tensor as (or otherwise aliases) one of its inputs under torch.compile, PyTorch now emits a UserWarning stating that this is deprecated and will become an error in a future version of PyTorch. Previously the warning stated the change would land in PyTorch 2.12; that timeline has been pushed back. To update your code, return a clone of the offending output instead of the input, or refactor the operator so it does not return the aliased tensor.

    Deprecated:

    @torch.library.custom_op("mylib::foo", mutates_args=())
def foo(x: torch.Tensor) -> torch.Tensor:
    return x  # output aliases the input -- deprecated

    Updated:

    @torch.library.custom_op("mylib::foo", mutates_args=())
def foo(x: torch.Tensor) -> torch.Tensor:
    return x.clone()  # return a clone instead

  • Creating tensors with the quantized dtypes quint8, qint8, and qint32 is now deprecated and emits a warning. This covers both Python and C++ call sites; see #184982 for migration guidance (#184984)

    PyTorch 2.12:

    >>> x = torch.quantize_per_tensor(torch.randn(3), 0.1, 0, torch.quint8)

    PyTorch 2.13:

    >>> x = torch.quantize_per_tensor(torch.randn(3), 0.1, 0, torch.quint8)
UserWarning: Creating tensors with quantized dtypes (quint8, qint8, qint32) is deprecated

  • Rename distributed collective ops to the _single naming scheme and deprecate the old names (#186123, #186124, #186125, #186134, #186135, #186144)

    To align the public torch.distributed collective APIs with the naming used by torchcomms' TorchCommBackend, all_gather_into_tensor is renamed to all_gather_single and reduce_scatter_tensor to reduce_scatter_single. The previous names continue to work as thin wrappers that delegate to the new functions, but now emit a FutureWarning.

    PyTorch 2.12:

    dist.all_gather_into_tensor(output, input)
dist.reduce_scatter_tensor(output, input)

    PyTorch 2.13:

    dist.all_gather_single(output, input)
dist.reduce_scatter_single(output, input)

New Features

Python Frontend
  • Add two new operator tags, torch.Tag.inplace and torch.Tag.out, that let an operator declare how it writes its result: inplace means it mutates a tensor in place, and out means it writes into a caller-provided output tensor. Native PyTorch operators are tagged automatically, and custom operators defined with torch.library can opt in by adding the tag. To be tagged inplace, an operator must take the tensor it mutates as its first positional argument (declared as Tensor(a!), and the only mutable argument) and return that same tensor. Tagging a custom operator this way improves its behavior under torch.compile: inplace ops now go through auto_functionalize, so the reinplacing pass can analyze clones and skip unnecessary copies, and both inplace and out ops get their fake/meta kernels generated for free. See the Python custom operators tutorial for how to author and tag custom operators. (#181100, #181099, #184199, #184200, #184201, #184202, #184203, #180851, #180852)
  • Add const_data_ptr() Python binding to torch.Tensor for read-only data pointer access (#180382)
  • Add an abbr property to torch.dtype that returns a dtype's short string abbreviation (e.g. torch.float32.abbr returns "f32") (#177296)
  • Allow positional arguments to be passed as keyword arguments to autograd custom Functions (#182206)
  • Expose rearrange in the torch.func namespace for einops-style tensor reshaping (#173183)
torch.nn
  • Add nn.LinearCrossEntropyLoss, a fused linear-projection plus cross-entropy loss module that avoids materializing the full logits tensor (#181573, #185852, #172286, #186113)
Autograd
  • Add torch.autograd.graph.region_activation_memory_budget (#185979)
  • Support passing gradient inputs as a dict to torch.autograd.grad and torch.autograd.backward (#178140)
Distributed

  • Add a registration API for symmetric memory arguments (lib.register_symm_mem_args()), letting operators (including out-of-tree ops) declare which arguments require symmetric-memory allocation (#173513)

  • Remove NCCLSymmetricMemory's explicit dependency on ProcessGroupNCCL, enabling symmetric memory to work with out-of-tree backends such as torchcomms (#184260)

  • Support accessing the ReduceOp.PREMUL_SUM factor from Python when implementing process group backends in Python (#185863)

  • Expose the NCCL 2.30 maxP2pPeers config binding (#181686)

  • Add rocSHMEM Triton integration for symmetric memory on ROCm (#178658)

  • Support passing extra keyword arguments to the loss function in pipeline schedules via a new loss_kwargs parameter to step(), enabling loss functions that require arguments beyond (output, target) (such as chunked cross-entropy needing token counts for scaling) (#181057)

Distributed FSDP2
  • Add FSDPModule.set_separate_reduce_scatter_group to give reduce-scatter its own NCCL communicator, enabling opt-in overlap of all-gather and reduce-scatter (#186335)
  • Add set_reduce_scatter_max_input_buffers to keep multiple reduce-scatter input buffers in flight, so backward compute no longer stalls waiting to recycle a single reduce-scatter buffer (#186000)
Profiler
  • Profiler/Kineto now emits channel metadata on CUDA backends (#185968)
Dynamo
  • Add torch.compiler.set_default_backend to override the default torch.compile backend globally, so out-of-tree backend authors don't need to pass backend= at every call site (following the pattern of torch.set_default_dtype/torch.set_default_device). Explicit backend= arguments still take precedence (#178944)
  • Add torch.compile(f, isolate_recompiles=True) to give each torch.compile call its own isolated cache bucket, preventing cross-compile interference in cache lookups and recompile-limit checks when multiple torch.compile calls target the same function (#178351)
  • Add register_multi_grad_hook support to @leaf_function, allowing a backward hook to fire once per backward pass when all requires_grad inputs have their gradients computed (#179609)
Inductor
  • Add flash-decoding support to the CPU FlexAttention template (chosen when query length is 1) with a new configurable PARTITION_SIZE kernel option (#159835)
  • Add Triton convolution backward kernels (input and weight gradients) as an autotuning backend in place of the ATen-only fallback (#178945)
  • Add an Inductor FX pass (decomp_comms) that eliminates all_gather for Gram-matrix optimizer patterns (Muon/Shampoo) under FSDP, gated by config.aten_distributed_optimizations.allow_comms_decompositions, yielding 1.25-1.95x training speedups (#184370)
Ahead-Of-Time Inductor (AOTI)
  • Generated C shims for the AOTI stable ABI are now versioned and gated by TORCH_TARGET_VERSION, so shims introduced in newer releases are only exposed when the target version supports them (#181916)
  • Triton CPU AOTI models now work end-to-end through the public torch._inductor.aoti_compile_and_package / aoti_load_package API, including packaging and loading of the multiple .so files emitted per kernel (#182251)
  • Added stable C shim functions (torch_exception_get_what, torch_exception_get_what_without_backtrace, and STABLE_TORCH_ERROR_CODE_CHECK) so extensions built against the stable ABI can retrieve the original error message across the C API boundary (target version 2.13+) (#180135)
  • Added a stable AOTI stream shim aoti_torch_stream_native_handle and torch::stable::accelerator::Stream::nativeHandle(), gated behind TORCH_FEATURE_VERSION >= 2.13, for retrieving a native stream handle from the stable ABI (#183930)
Release Engineering
  • Add Python 3.15 wheel builds across Linux (CPU/CUDA), Triton, ROCm, and XPU (#182954, #184600, #185409, #184829, #184891, #184906, #185094)
CUDA
  • Add CUDAGraph.get_graph_data() for graph topology introspection (#183165)
  • Lightweight API to get private pool reserved memory bytes (#178240)
MPS
  • Add FlexAttention support for MPS (#182552, #186215)
  • Add support for torch.distributions.Dirichlet on MPS by adding _sample_dirichlet and _dirichlet_grad Metal implementations (#185458, #185854)
  • Add grid_sampler_2d backward support on MPS (#179756)
  • Add grid_sampler_3d backward support on MPS (#179388)
  • Add lcm support on MPS via a new Metal kernel (#186279)
  • Add complex support to c10/metal/reduction_utils.h (#180708) and a complex->bool specialization (#185938)
ROCm
  • Enable external events in CUDA graphs (#178264)
  • Enable GPU Address Sanitizer build (#183792, #176461)
  • Improve Inductor GEMM search space performance using the Origami project (#172512)
  • Use CMake native HIP language support, enable_language(HIP) (#180485)
  • New Inductor benchmarker based on Torch Profiler (#175097)
XPU
  • Add XPU device telemetry APIs for temperature, frequency, power draw, engine utilization, memory bandwidth usage, and used device memory through torch.xpu.* (#181082, #183427, #183428, #183429, #183430, #183431)
  • Add FP8 blockwise scaling support for scaled_mm on XPU (#173630, #176043)

Improvements

Python Frontend
  • Make it possible to load safetensors with torch.load (#170592)
  • Make Storage.pin_memory / Storage.is_pinned device-agnostic (#186223)
  • Add op_overloads to OpOverloadPacket to enumerate an operator's overloads (#182993)
torch.nn
  • Expose num_splits in FlashAttention-2 and bump the flash-attention submodule (#179760)
  • Support linear_bias in linear_cross_entropy on the reference and chunked paths (#185129, #185276)
Optimizer
  • Fix SequentialLR wrong learning rate initialization when milestones contain 0 (#185986)
Autograd
  • Implement autograd derivatives for torch.nextafter (#148820)
  • Add torch.autograd.enforce_grad_layout_policy to control the memory layout policy for accumulated gradients (#180552)
Distributed

  • When TorchComms is enabled, route new_group through split_group for subgroup creation, raising NotImplementedError for arguments split_group cannot honor (e.g. use_local_synchronization=True, sort_ranks=False) instead of silently falling back (#185416)

  • Delegate dist.new_group to custom process group subclasses (#184262)

  • Surface started-work metadata in NCCL watchdog timeouts (#183656)

  • Add a health check endpoint to the distributed debug server (#179326)

  • Make the DeviceMesh non-overlapping check stricter (#172343)

  • Allow elastic_launch/launch_agent to accept a pre-created torchelastic health check server, so it can be started before rendezvous (#180543)

  • Add an overlap_pp_comm flag to pipeline schedules (default True) that, when set to False, defers each pipeline RECV op to immediately before the compute op that consumes it, using rank-parity P2P ordering to avoid deadlock (helps platforms such as AMD ROCm where a pending RECV blocks unrelated compute) (#178815)

DTensor
  • Migrate embedding and random ops to single-dim sharding strategies and increase op coverage (#180281, #180503)
  • Add auto-infrastructure that derives single-dim sharding strategies for autogenerated op variants (.out, inplace, functional, and foreach), expanding strategy coverage to hundreds of additional ops (#185386)
  • Register sharding strategies for additional ops: scatter, upsample/interpolation backward, anti-aliased upsample, batch norm backward, and aten.detach_.default (#186149, #180311, #184626, #182743, #181876)
Distributed FSDP2
  • Support forward-mode automatic differentiation (torch.func.jvp) on models wrapped with fully_shard or replicate, including with mixed precision (#182732)
Linear Algebra Frontend
  • Add Half and BFloat16 dispatch support for torch.trace on CPU (#184874)
  • Improve heuristics for the cuSOLVER vs cuBLAS backend switch in torch.linalg.lu (#185344)
Profiler
  • The memory viz tool now more accurately represents GPU footprint when impacted by fragmentation (#180515)
  • The memory viz tool now aggregates stripes per-pool to improve visualization for large snapshots (#180613)
  • Profiler now also exposes CUDA occupancy metadata as a nested dictionary in the .events() output (#180275)
FX

  • split_module now supports torch.Size crossing graph split boundaries by decomposing size() calls into per-dimension sym_size nodes, and builds submodules lazily for faster inference graph splitting (#179839)

  • CapabilityBasedPartitioner can now opt out of horizontal fusion via skip_horizontal_fusion=True, partitioning only through direct data dependencies (#184904)

  • Enable rewriting of FX traces containing complex tensors during compilation (#169832)

Dynamo
  • Implement additional Python operators in Dynamo: bitwise and (#184788), bitwise xor (#184789), left/right shift (#183462), floor division (#185652), true division (#185653), remainder (#185654), and divmod (#185655)
  • Support tracing more constructs in Dynamo: einops 0.8.2 (#185619), record_function as a decorator (#184703), inference_mode retracing helpers (#185066), mark_dirty in the autograd Function HOP (#184267), warn_only deterministic toggles (#180373), and the _maybe_view_chunk_cat functional collective (#180389)
  • Support item assignment and deletion (__setitem__/__delitem__) on more container types in Dynamo via sq_ass_item/mp_ass_subscript slots (#182862, #182996)
  • Support torch.accelerator.device_index and torch.xpu.device in the device context manager (#181846, #181847)
  • Improve Triton support under torch.compile: accept tl.constexpr values as kernel arguments (#181783) and handle capture_triton as a no-op during tracing (#183555)
  • Improve dynamic shape specification: reduce verbosity in shape specs for the common case (#184271), add SeqSpec for list/tuple specs with better walk-spec errors (#185327), add ObjectSpec (#182764), pipe dynamic spec through torch.compile (#184501), and revisit guarding in mark_dynamic APIs (#181469)
  • Improve torch.compile device mismatch errors with a dedicated FakeTensorDeviceMismatchError and actionable guidance to place inputs, parameters, and buffers on the same device (#185412)
  • Improve error messages and diagnostics: clearer data-dependent errors for .any()/.all() (#180406), clearer torch._check tensor predicate errors (#185777), user-friendly reasons for skipped frames (#183596), carets in stack traces (#182393), and reporting why a symbol was created dynamically in symbolic_shapes logs (#168331)
  • Make Dynamo exceptions pickleable (#185725)
  • Inline decomposed quantization helpers in Dynamo (#185628)
  • Make Dynamo debug/repro utilities device-agnostic (#184851)
Inductor
  • Support pin_memory for torch.ones, torch.zeros and torch.full (#174595)
  • Add a ROCm config flag to disable the pointer_range_32 optimization (#179604)
  • Add a peak memory threshold config for combo kernels (#180578)
  • Enable autotuning and a fast compensation path for CPU static/dynamic smooth-quant qlinear, with correct handling of 0D x_scale/x_zp and ReinterpretView strides (#181090)
  • Add aot_inductor.autotune_per_kernel_alloc config to allocate-run-delete tensors per kernel during AOTI autotuning, avoiding OOM on large models (#181176)
  • Bound AsyncCompile future waits with the compile_worker_wait_timeout setting (#181293)
  • Add a100_default_flex_config entries for head_dim=192 (#181835)
  • Add an explicit lowering (fallback) for aten.multinomial to avoid graph breaks (#182423)
  • Add an Inductor lowering for _scaled_mm_v2 (#182527)
  • Enable combo_kernel_autotune_grouping by default (#182567)
  • Add cudagraph_partition_memory_budget config for partition reordering (#183569)
  • Add ATen fallbacks for bincount, unique variants, and AMP scale ops so they compile without graph breaks (#183590)
  • Unfuse the bias add from addmm when the bias is a narrowing dtype cast (fp32->bf16/fp16) to preserve bias precision in XPU AMP training (#183680)
  • Emit a clearer diagnostic when a backward CUDAGraph output installed as a .grad buffer is invalidated on a later run (gradient accumulation) (#184003)
  • Support split online softmax reductions in Inductor (#184069)
  • Support fusing index_add-style atomic scatter mutations into Triton template epilogues behind a config flag (#184179)
  • Add a cpp.march Inductor config knob so AOTInductor cpp-only builds can override or suppress the default CPU architecture flag (#184297)
  • Improve the Triton cache directory guidance when loading shared objects from a noexec filesystem fails (#184362)
  • Enable the Bert SDPA pattern rewrite on CUDA while keeping the original matmul/softmax math path (#184417)
  • Improve the Triton launcher argument-mismatch error with a clearer message and a cached preflight check (#184522)
  • Add broader CuteDSL op overrides (rsqrt, exp2, log2, log10, tan, acos, asin, atan, atan2, floor, logical_xor) (#184538)
  • Add an optional fake_mode argument to standalone_compile (with dynamic_shapes="from_example_inputs") so it can reuse the caller's FakeTensorMode/ShapeEnv instead of always creating a fresh one (#184776)
  • Support signbit on unsigned integer dtypes (#185985)
  • Add a keep_static_cubin_raw config to retain cubin bytes in cached kernels so caches restored on another machine avoid recompilation (#186404)
  • Extend BatchLinearLHSFusion's matcher to also match the inlined torch._C._nn.linear form so the (opt-in) fusion can fire on Dynamo-inlined linear (#186632)
  • Add a clearer error message with install instructions when a compatible Flash Attention package is unavailable for flex attention (#186827)
  • Add another anchor node to the batch-linear fusion pass so more small torch._C._nn.linear operations are grouped into a single batched kernel (#180477)
  • Extend the batch fusion pass to support detach() method calls (#180513)
  • Add a deterministic backward for the FlexAttention flash kernel (#174813)
  • Use rand4x for Inductor Triton random number generation (#184377)
  • Add a Quack-based CuTeDSL RMSNorm kernel (#182108)
Ahead-Of-Time Inductor (AOTI)
  • Use fatbinary for multi-arch CUDA kernels (#184456)
  • Support mixed-device constants in update_constant_buffer (#181114)
  • Add FP8 header files in the AOTI shim.h (#178120)
  • Add throttled cudaMemcpy for AOTI constant loading to reduce peak memory usage (#184823)
  • Preserve AOTI proxy_executor error messages (#180884)
  • Enable Triton kernels in AOTI C++ wrapper on CPU (#181068)
  • Skip CPU vec ISA setup for device-only cpp_wrapper (#182089)
  • Expose torchbind constants from AOTIModelPackageLoader (#182149)
  • Improve AOTI error for Python custom ops (#186305)
Export
  • Support serialization of opaque type constants in torch.export save/load (#181676)
  • Make functorch JVP operator torch.export-able (#179686)
  • Add UpdateConstantBufferFromCpu for host-to-device copy (#181637)
AOTAutograd
  • Support CPU activation offloading in the rematerialization pass, including marking recomputed nodes for backward and adding a resize-to-0 deallocation op so offloaded tensors are freed after their host-to-device copy (#181937, #181938)
  • Use FX node names in merge_view_inputs error messages, so non-differentiable view input mutation errors identify the specific offending inputs (#180424)
Composability
  • Add fake tensor support for _transformer_encoder_layer_fwd so it traces under torch.compile (#183916)
  • Enable Armv9-A target support for torch.compile on AArch64 (#184555)
  • Functionalize in-place c10d collectives in standalone compile (#181836)
Foreach
  • Fix/add empty check for _foreach_max (#173483)
ONNX
  • Add adaptive_max_pool2d and adaptive_max_pool3d decompositions for ONNX export (#184396)
C++ Frontend
  • Add stable ABI for set_python_module on torch::Library (#182720)
  • Add == overloads for HeaderOnlyArrayRef (#185379)
  • Add torch::stable::Generator (#186423)
  • Add c10::layout typecaster for torch.layout (#179607)
  • Add default-args support to def_static (#175644)
  • Add support for controlling scientific notation in C++-side tensor printing (#173321)
Release Engineering
  • Move the NCCL pin to 2.30 (#181313)
  • Advance the Triton pin to 3.7.1 (#181001, #186792)
  • Upgrade the XPU support package to 2026.0 (#182003)
  • Add a configurable threshold to avoid power-of-two rounding for large pinned memory allocations (#171662)
  • Move some pre-build steps from setup.py to CMake (#177641)
CUDA
  • Debugging tool to verify that external inputs to a CUDA graph are alive before replay (#174649)
  • Add get/set/reset functions for BLAS workspace sizes (#177912)
  • Cleanup double import in BinaryDivFloorKernel.cu (#179260)
  • Return supported CUDA arch list when no GPU is present but GPU is compiled (#180356)
  • Detect and fix stale stream references in autograd during CUDA graph capture (#180090)
  • Use opmath_t in i1 and i1e CUDA kernels (#183778)
  • Support resize_ with address hint (#178215)
  • Support bfloat16 in _embedding_bag_per_sample_weights_backward on CUDA (#185889)
  • Align parsePerProcessMemoryFraction's return type with other parsers (#185139)
  • Improve error message when cuda-bindings version is too old (#185990)
  • Expose torch.cuda.current_solver_handle for cuSOLVER handle sharing (#176705)
cuDNN
  • Add flag to select depthwise convolution backend (#176500)
  • Upgrade cudnn_frontend submodule to 1.24 (#185554)
MPS
  • Migrate many ops from MPSGraph to native Metal kernels for bernoulli (#182210), native_dropout (#182232), uniform/normal/randint (#182386), randperm (#182528), comparison ops eq/ne/lt/le/gt/ge (#183019), bitwise ops (#182839), scatter/gather (#184028), copy-cast (#184740), gelu/gelu_backward (#181451), replication pad (#183065), embedding backward (#185119), trace (#183627), count_nonzero (#180725), amax/amin/aminmax/all/any (#180752), and cumsum/cumprod (#185609), native_group_norm (#183830, native_group_norm_backward #184437), topk and kthvalue Metal kernels (#184106), single-block and multi-block sort, including a stable sort path (#180714, #182242, #181736)
  • Support integer inputs to histc (#178624)
  • Support out variants of unary ops (#184743)
  • Improvements to SDPA Metal kernels: prefill attention kernels (#181575), GQA support (#183280), is_causal support (#181855), head dim 256 (#181852), float mask support (#183458), and a clearer error for causal + attn mask (#181856)
  • Add an ILP variant for binary tensor iterators and replace unary VEC4 with a generic ILP-per-thread dense kernel (#182155, #181509, #183055)
  • Make several Metal kernels stride-aware to avoid unnecessary .contiguous() calls: Col2Im (#181949), Im2Col (#182709), Repeat (#182718), LossOps (#182714), and HistogramKernel (#181951)
  • Enable NDHWC+DHWIO fast path for Conv3d on channels_last_3d (#184612)
  • Clear the MPSGraph cache in torch.mps.empty_cache() (#181485)
  • Add a private API to get the host alias of Metal storage (#180961)
  • Add input validation: stride > 0 in pool ops (#184875), F.fold (#182067), im2col (#183593), and bernoulli probabilities (#185065)
  • Alert on non-deterministic algorithms on MPS (#185061)
  • Raise a clear not-implemented error for dropout_p (#184126)
  • Use proper precise log functions in unary kernels (#185381)
ROCm
  • Additional cub::DeviceHistogram hipify mappings (#180433)
  • SDPA improvements via AOTriton 0.12b: head_dim != head_dim_v, use_deterministic_algorithms, gfx1100 and gfx1151 promoted out of experimental, partial FAv3 support on gfx950 (#184288)
XPU
  • Add last_level_cache_size and is_integrated_gpu to XPU device properties (#184499, #182624)
  • Add XPU dispatch for _fused_adagrad_ (#185577)
  • Support mixed-type operations between Nested and Dense tensors on XPU (#182654)
  • Support torch.xpu.device in Dynamo device management (#181847)
  • Recognize additional Intel BMG device IDs on XPU (#183414)
  • Enable XPU device support for sparse Triton ops (#179805)
  • Enable the bmm_outer_product Triton override on XPU (#180441)
  • Improve test coverage for the XPU backend (#174370, #180881, #171154)
  • Support non-blocking pinned device-to-host copies on XPU (#186224)
  • Refactor the XPU oneDNN integration from the C API to the C++ API (#184486)
Functorch
  • Add vmap batching rule for torch.unbind_copy (#178035)
  • Add vmap batching rule for Tensor.view(dtype) (aten::view.dtype) (#180728)
Sparse Frontend
  • Validate mat1/mat2 layouts in sspaddmm and clarify error messages (#179037)

Bug Fixes

Python Frontend
  • Add opt_dtype validation to torch.nanmean() for consistent error handling (#172809)
  • Route torch.nansum integer output dtype through nan_to_num + sum for correct results (#183808)
  • Fix out-of-bounds read in CUDAStream::stream() (#184237)
  • Align XPU logspace/linspace ref tests with upstream XFAIL state (#178734)
Dataloader Frontend
  • Fix DataLoader file descriptor leak from atexit cleanup (#176607)
torch.nn
  • Validate stride/padding/kernel_size length in slow_conv3d (#181063)
  • Validate inputs in math_channel_shuffle (#181029)
  • Validate delta type in nn.HuberLoss constructor (#184012)
  • Lowercase the environment variable in torch/serialization.py so it matches the true values (#180959)
  • Fix int32 overflow in layer_norm on CUDA for tensors with more than 2^32 elements (#181600)
  • Reject NestedTensor inputs in flex_attention with a clear error instead of an unclear backend failure (#183516)
  • Fix SDPA incorrect early return on 0 head dim qk with valid v (#184914)
  • Fix reflection_pad1d backward CUDA launch for large batches (#185024)
  • Fix lp_pool infinity norm handling (#183997)
Autograd
  • Fix checkpoint context cleanup on forward errors (#184018)
  • Fix torch.autograd.enforce_grad_layout_policy decorator state leak (#183868)
Distributed

  • Fix NCCLComm::abort() to use the correct deregister API for window-registered handles (#181626)

  • Fix FakeProcessGroup all_gather on tensors that require grad (#181790)

  • Fix gather and allgather_coalesced on FakeProcessGroup to copy input to output (#182364)

  • Fix the scatter and reduce_scatter family on FakeProcessGroup to copy input to output (#182365)

  • Fix all_to_all on FakeProcessGroup and validate splits (#182366)

  • Fix conflict between broadcast_buffers and init_sync in DDP (#178054)

  • Fix gather on non-destination ranks for the TorchComms backend (#178533)

  • Fix TCPStore compilation with Clang 20 (#185785)

  • Fix NCCL symmetric memory mismatch by using an allocation-time counter instead of address for block ordering (#183489)

  • Fix a symbol lookup issue with the symmetric memory __init__ (#186416)

  • Fix the value returned by Work.exception() so the exception can be inspected from Python instead of being unusable (#184697)

  • Fix false assertion errors in the flight recorder when using the ncclx, gloo, rccl, rcclx, mccl, or hccl backends (#179753)

  • Fix a failure when creating a subgroup on a fake backend via new_group, which has no underlying communicator to split (#186172)

  • Fix torch.compile of the _c10d_functional all_gather_tensor_out and reduce_scatter_tensor_out ops, which previously failed functionalization with "Found a custom (non-ATen) operator whose output has alias annotations" (#183597)

  • Fix split_group on multi-backend process groups (e.g. init_process_group(backend="cpu:gloo,cuda:nccl")) to split only the relevant backend instead of every backend, avoiding spurious warnings, extra rendezvous overhead, and inconsistent process-group shapes (#182057)

  • Fix FakeProcessGroup to reject rank >= world_size at construction time, which previously failed silently and only surfaced later when collectives indexed past world_size (#182363)

  • Fix the torchelastic agent hanging indefinitely (and never exiting) when workers become stuck in an uninterruptible (D-state) process that SIGKILL cannot reap; the final proc.join()/proc.wait() in _close is now bounded by a timeout and the unkillable PID is logged (#185414)

  • Fix pipelining producing incorrect results or cryptic runtime errors when a PipelineScheduleMulti topology communicates between non-adjacent stages (e.g. skip connections); this is now detected at initialization and raises a clear RuntimeError (#179293)

  • Fix RuntimeError: only Tensors of floating point dtype can require gradients when building a pipeline for models with non-float intermediates (such as Hugging Face transformer models) (#183582)

  • Make LocalTensorMode transparent to torch.compile so compilation proceeds as if the debugging mode were not active (#182667)

  • Fix AssertionError in elastic c10d rendezvous when a node's rank changes across rendezvous rounds (e.g. a node becomes rank 0 after a peer leaves) (#182375)

  • Fix shared-weight gradient double-counting in zero-bubble pipeline schedules (#181365)

  • Fix None gradient handling in pipeline backward send/recv (#182182)

  • Fix a pipelining crash when split_module interleaves get_attr nodes with placeholder nodes (#182644)

Distributed Checkpointing
  • Fix optimizer state-dict save/load using the wrong process group for FSDP models initialized with a non-default process group, by forwarding the FSDP process group to FSDP.optim_state_dict and FSDP.optim_state_dict_to_load (#181261)
  • Fix DefaultStager crash when reused (#183424)
DTensor
  • Fix Context Parallel load balancing for short sequences that cannot be split into the required head/tail chunks, falling back to regular CP sharding for correctness (#183968)
  • Fix squeeze leaving a DTensor's spec and local tensor out of sync (spec shape not matching the local tensor) by preventing squeeze from redistributing when strict_view is set (#175798)
  • Fix .view() failures (e.g. in ColwiseParallel/RowwiseParallel) after an uneven Shard(dim>0)->Replicate redistribute by making the local tensor contiguous (#184443)
  • Fix a device-side assert crash when tracing tensor-parallel models with make_fx, caused by sharding propagation recording dead global-shape shadow nodes into the graph (#185865)
  • Avoid the native sharding cache for symbolic IValue arguments, fixing stale cached output shardings that confused global and local shard shapes for shape-sensitive ops under compiled autograd (#183246)
  • Fix upsample backward crashes when the input is sharded (#182595)
  • Fix Partial placement being lost during the autograd layout invariant (#180511)
  • Fix OpSpec.mesh crash when specs contain None entries (#181541)
  • Fix redistribute(backward_dtype=...) ignoring the backward dtype (#182032)
  • Fix _StridedShard flag conflict during gradient accumulation (#183517)
  • Fix reduction strategy linearity (#183794)
  • Fix the cache key hashing for fake meshes (#184001)
  • Fix FakeTensor device hint in sharding propagation (#183970)
  • Fix group_norm scalar adjuster crash when weight=None (#184819)
  • Fix to_local() dropping the _is_param marker that nn.Parameter sets on custom tensors (#184422)
  • Fix pad_tensor/unpad_tensor creating unnecessary guards on symbolic pad sizes during tracing (#180887)
  • Fix DTensor + activation checkpointing + torch.compile crash caused by an unbound inner symbol at the root tracer (#181797)
  • Fix a DTensorSpec refcount leak in OpSchema._recompute_comparison_key (#181792)
Distributed FSDP2
  • Fix stale post_accumulate_grad_hook results under CPUOffloadPolicy (#180666)
  • Fix reduce-scatter of unused DTensor parameters that previously raised a mixed Tensor/DTensor error in chunk_cat (#183040)
  • Fix IndexError for modules called with no forward inputs by preserving empty args/kwargs, matching FSDP1 behavior (#183943)
  • Remove redundant stream waits (#183983)
  • Fix a tensor-parallel + FSDP2 + mixed-precision bug where the unsharded compute tensor was wrapped back into a DTensor with a stale fp32 dtype, causing sharding propagation to fail in eager and torch.compile (#183805)
  • Fix incorrectly reduced gradients when running a partial forward of a fully_shard([norm, head]) group for chunked loss, where the model forward runs norm only and head is called standalone per chunk; unshard/reshard previously relied on _modules_to_run_forward and produced wrong results for this pattern (#180428)
  • Add a warning when a grad-requiring forward output is a view tensor, since in-place ops on the view silently drop the pre-backward hook and cause backward to skip the all-gather and fail (#181850)
  • Fix incorrect clip_grad_norm results with multiple data-parallel shard axes (e.g. dp_shard + cp passed via DataParallelMeshDims), which exposed separate Shard axes with the wrong float32 reduction order and an incorrect Shard instead of _StridedShard; the axes are now flattened into a single shard axis in the sharding spec (#183629)
  • Fix recomputed tensor metadata diverging from the original forward under activation checkpointing when cast_forward_inputs is enabled, by casting forward inputs during recompute (#182580)
  • Fix incorrect resharding of full-DTensor parameters when tensor parallelism is also applied by using _StridedShard (#186126)
Linear Algebra Frontend
  • Validate pivot range in torch.linalg.ldl_solve CPU kernel (#181032)
  • Fix rocBLAS tunable GEMM solution handling in TunableOp (#182380)
  • Route fp16 backward GEMMs to rocBLAS to preserve subnormals (#183766)
  • Make collective bucketing tolerate hinted unbacked SymInts (#183544)
Profiler
  • Fix an issue where profiler would issue a "Profiler clears events at the end of each cycle" warning even when no cycles are used in the schedule (#180387)
  • Ensure that Profiler does not keep driving Kineto transitions even when GPU collection has stopped (#180698)
FX
  • Preserve user runtime asserts in FX pass (#184608)
Dynamo
  • Fix Python container and operator semantics in Dynamo to match CPython: list (#185425) and tuple (#185427) constructors, dict update (#185428), defaultdict inplace union (#185429), frozenset copy identity (#185430), sequence search (#185431), iand on bool constants (#184503), sequence * SymNode spurious graph break (#185260), and torch.Size tensor shape handling (#184613)
  • Fix symbolic shape / fake tensor handling: float/bool + SymNode (#183362), PendingUnbackedSymbolNotFound for 0-d tensor Scalar args (#182660), GuardOnDataDependentSymNode on sparse tensors (#179616), and creating symbolic tensors from foreign fake tensors (#181794)
  • Fix use-after-free issues in CUDAStream/Event tp_dealloc overrides (#183403) and Dynamo dict guard cleanup (#183753)
  • Fix Dynamo crash when DeviceMesh is constructed inside torch.compile (#177201)
  • Fix torch.compile crash when an unsupported type is passed to a tensor method inside try/except (#182106)
  • Skip wrap_inline for exec'd Python functions (#181531)
  • Fix tensor subclass construction under torch.compile (#183337)
  • Preserve eager torch.full validation for nn.Parameter fill values in Dynamo (#183915)
  • Prevent accuracy minifier repro recursion (#184077)
  • Fix AOT export with flex attention BlockMask placeholders (#184611)
  • Accept extra kwargs in CudagraphsBackend.__call__ (#182989)
  • Avoid def forward(self, ..., self, ...) SyntaxError in dynamo_graph_capture_for_export (#185314)
  • Fix scoped Chromium event reset (#184973)
  • Fix Dynamo binding of overridden function defaults (#184852)
  • Fix SAC context_fn clobbered by DDPOptimizer's propagate_metadata (#179496)
  • Clear retained fake tensor CUDA constants (#184445)
  • Fix Dynamo .grad reads for new in-graph parameters (#184972)
  • Graph break on CUDA manual_seed in Dynamo so compiled random calls stay reproducible (#185761)
  • Fix functional tensor to_dense no-op (#184586)
  • Preserve original tensor strides across activation offload/reload (#186396)
  • Graph break on duplicate autograd Function inputs (#184281)
  • Raise IndexError in compile mode matching eager mode (#184856)
  • Don't error on a skipped frame when fullgraph=True and a non-default stance is set (#183623)
  • Set the is_compiling flag for the whole torch.compile session (#184614)
  • Fix FxGraphCache pickling of opaque types with cyclic references (#180422)
  • Handle missing Windows C++ compiler in shape guard fallback (#185447)
Inductor
  • Fix max_autotune BMM correctness with dynamic OpenMP threads (#169128)
  • Fix NaN output in CPU LayerNorm by guarding the Welford variance computation (#173989)
  • Fix NameError from Python name mangling for user-defined Triton kernels whose names start with double underscores (#176100)
  • Fix backward-pass shape mismatch in mix-order reduction for compiled models (#178098)
  • Do not apply pointer_range_32 to user-defined Triton kernels on ROCm to avoid compilation crashes (#178541)
  • Fix the e8m0_rceil_log2 pattern failing to register on any CUDA device (#178698)
  • Fix cpp_wrapper backward compilation failing with a CudaKernelParamCache assertion when set via torch.compile options (#178847)
  • Fix FxGraphCachePickler crash on unpicklable pybind11 extension types during cache key computation (#178853)
  • Fix C++ compile error when indexing a transposed tensor with an indirect (tensor) index on the CPU backend (#178962)
  • Fix TPU Pallas codegen to use subscript indexing for scalar and size-1 buffer accesses (#179099)
  • Fix incorrect argmax/argmin indices in the CPP Tile2D vectorized kernel for transposed inputs (#179525)
  • Fix user-kernel fusion to use mutation_outputs for the intermediate buffer and add epilogue source to the cache key (#179803)
  • Fix missing mask on tl.atomic_add for constant-index stores (#179833)
  • Add convolution input/weight dtype-mismatch check to match eager semantics (#179890)
  • Raise a catchable error instead of crashing on integer divide-by-zero for fmod/remainder on CPU (#179923)
  • Fix stride mismatch for user-visible reduction nodes caused by dislike_padding conflicting with contiguous storage layout (#180197)
  • Fix check_bounds forward-reference C++ compile error in the CPU backend (#180212)
  • Preserve Triton HIP compile options (waves_per_eu, matrix_instr_nonkdim, kpack) when combo kernels rewrite per-subkernel configs on ROCm (#180277)
  • Fix torch.compile crash on cumsum with broadcast input when the scan dimension is 129 or larger (#180369)
  • Fix AOTInductor while_loop codegen for the minimal-arrayref interface (wrap ArrayRefTensor inputs before assign) (#180370)
  • Fix SM100 addmm CUTLASS codegen and relax addmm tolerances (#180432)
  • Fix incorrect storage-offset propagation for as_strided (#180581)
  • Filter CUTLASS kernels by fast-accum only on Hopper to avoid NoValidChoicesError for scaled_mm/scaled_grouped_mm with use_fast_accum=True on Blackwell (#180586)
  • Make persistent-reduction config selection batch-invariant under deterministic mode (#180832)
  • Fix UnicodeDecodeError on Windows when using icx-cl as the CXX compiler by decoding subprocess output with errors='replace' (#180853)
  • Avoid emitting tl.float64 in Triton scalar-shape-math codegen on XPU devices without fp64 support (e.g. Intel Arc A770) (#180854)
  • Fix silent incorrect results when adaptive_avg_pool2d is fused with downstream flatten + reduction for non-power-of-2 channel sizes (#180898)
  • Fix CPU C++ fusion crash on GroupNorm + SDPA + bmm (#181015)
  • Fix num_warps on AMD RDNA (wave32) GPUs so it is no longer incorrectly halved; only CDNA (warp_size 64) is halved (#181112)
  • Fix unsupported index_expr/kindexpr error when using flex_attention with bias/index expressions (#181207)
  • Fix unbacked-symbol bindings assertion in the slice lowering early-return path (#181219)
  • Preserve program order of random ops in the scheduler to avoid numerical mismatch vs eager when fallback_random is enabled (#181245)
  • Fix Inductor crash (NotImplementedError: View) when a flatten()-produced view is passed as the conv2d bias (#181363)
  • Include per-device autocast dtype in the AOTAutograd cache key to prevent reusing a graph compiled under one autocast dtype (e.g. bfloat16) when running under another (e.g. float16) (#181564)
  • Fix _to_copy decomposition so float64->float32 transfers to XPU devices without fp64 support no longer generate illegal fp64 buffers/crash (#181607)
  • Prevent AutoHeuristic initialization from crashing in non-CUDA (e.g. XPU) environments (#181745)
  • Fix AttributeError crash in remove_noop_ops when a node's meta val is a SymInt/SymFloat (auto dynamic shapes) instead of a Tensor (#181752)
  • Fix ConcatKernel channels_last contiguous check to handle symbolic shapes, avoiding a data-dependent (DDE) LoweringException (#181845)
  • Fix user-defined Triton kernel pointer args with no tl.load/tl.store being incorrectly eliminated (and not allocated) when epilogue fusion is enabled (#181868)
  • Properly truncate (downcast then upcast) the accumulator for Triton GEMM epilogue fusions so numerics match eager (#181918)
  • Skip non-tensor IR nodes (e.g. process groups) when realizing/marking mutated buffers in _CollectiveKernel.create_inplace (#181930)
  • Pass the correct device to print_performance in generated benchmark_compiled_module so timing works on non-CUDA devices (e.g. XPU) (#181957)
  • Serialize CompiledFxGraph._original_gm via GraphPickler to fix AOTAutograd/precompile cache serialization crashes for graphs with HOPs that have lifted buffers (e.g. flex_attention with a causal BlockMask) (#182088)
  • Avoid raw stream name collisions in generated Inductor code (#182139)
  • Disable the overlap-scheduling SPMD check by default to avoid an NCCL hang when ranks diverge on cache hit vs miss (#182281)
  • Fix CUDA graphs to support self-overlapping inputs and unbacked shapes via a storage-copy fallback instead of erroring on complex memory overlap (#182524)
  • Pass derived sym-int captures through maybe_realize in FlexAttention (#182610)
  • Fix AutoHeuristic crash in non-CUDA environments (e.g. XPU and CPU-only builds) (#182614)
  • Do not automatically move non-deterministic seed functions to GPU (#182748)
  • Fix emulate_precision_casts on CPU C++ codegen so explicit/emulated fp32->fp16->fp32 precision barriers are no longer optimized away (#182882)
  • Fix block-descriptor final shape expansion for discontiguous fused pointwise/reduction loads and stores (#182936)
  • Fix crash in FLOP counting (count_flops_fx) for HigherOrderOperator targets such as flex_attention during overlap/bucketing (#182992)
  • Raise device-mismatch errors for index_add/index_copy/index_reduce instead of silently succeeding under torch.compile (#183007)
  • Fix Triton eager_input_vals propagation (#183334)
  • Escape backslashes in user-defined Triton kernel source (#183421)
  • Fix TMA config lowering block sizes below the heuristic choice, which could trigger an illegal memory access on large tensors (#183438)
  • Fix CPU AOTInductor with user-defined Triton kernels when compile-time autotuning is disabled (#183463)
  • Fix is_linear_add_bias crash when the bias node argument is a Python float (#183514)
  • Fix convolution lowering and backward decomposition crash when a channel dimension is zero (#183539)
  • Fix Inductor embedding negative index checks (#183636)
  • Disable cudagraphs for input storage mutation (#183645)
  • Fix torch.compile crash on cumprod backward by disallowing fusion of split-scan nodes with reductions (#183653)
  • Fix CPU argmax logical index for transposed reductions (#183655)
  • Fix while_loop backward expanded gradient strides (#183658)
  • Fix the SDPA backward constraint for scalar gradient bases (#183662)
  • Fix Inductor multi-stream codegen on XPU to emit torch.xpu stream APIs instead of CUDA-only calls (#183693)
  • Fix BF16 Inductor failure on Neoverse V3 with GCC 12.4 (#183698)
  • Fix the tail reduction suffix width (#183699)
  • Support unused lifted SymInts in associative_scan lowering (#183706)
  • Guard the fast Triton launcher to CUDA/HIP so XPU correctly falls back to the static launcher (#183707)
  • Fix CPU C++ codegen crash from misaligned loop-split groups after fusion (#183715)
  • Fix set_.source_Tensor lowering when the source is a view (#183724)
  • Fix CPU GEMM k-slicing cache-block indexing (#183733)
  • Skip cudagraph capture when the CUDA caching allocator is bypassed (#183780)
  • Fix bytes-vs-elements mismatch in select_algorithm input storage check (#183791)
  • Use libdevice log for strict Inductor numerics (#183844)
  • Add meta registration and dtype check for div_ to match eager type-promotion semantics (#183859)
  • Fix handle_synced_deallocation for XPU (#183865)
  • Use hipModuleLoadData in StaticCudaLauncher on ROCm (#183926)
  • Fix flex_attention autotune logging to use size hints for symbolic dims, avoiding lowering failures under dynamic shapes (#183933)
  • Guard the SM100 E8M0 FP8 PTX lowering to NVIDIA so ROCm gfx11 uses the portable bit-manipulation fallback (#183949)
  • Enable the Python dispatcher when re-running fake propagation for fallback kernels so custom/backend fake implementations supply correct output layout metadata (#183961)
  • Fix a crash in tuned_addmm with max-autotune when the bias is an unrealized view (e.g. a transposed expression) (#183973)
  • Compute gelu backward in opmath dtype for improved numerical accuracy (#183985)
  • Cap Triton convolution warps for non-1x1 kernels to avoid miscompilation (#183989)
  • Fix graph partition signatures to only drop buffers removed during that partition's codegen, preserving globally-removed buffers still needed by later partitions (#184004)
  • Handle float8_e4m3fn dequantization on pre-sm89 CUDA devices to match eager bit patterns (#184008)
  • Fix the native matmul config numel cap (#184011)
  • Use CUDA libdevice for emulate-precision casts to match eager numerics (#184022)
  • Fix Inductor compile-worker shutdown so the sidecar's nested worker pool is terminated and shutdown no longer hangs until the parent wait timeout (#184038)
  • Fix bucketize/searchsorted on sliced or zero-stride lookup tensors (#184043)
  • Fix scaled softmax non-finite semantics (#184046)
  • Fix CPU FlexAttention aliased input lowering (#184071)
  • Rewrite Triton floating-point self-subtraction to produce exact zero (#184093)
  • Treat corrupt local autotune cache entries (bad JSON/invalid UTF-8) as cache misses instead of crashing compilation (#184096)
  • Recurse through list/tuple arguments when applying needs_fixed_stride_order so fallback custom ops receive tensor-list inputs in the recorded stride order (#184098)
  • Fix CPU Inductor vectorized asinh overflow (#184105)
  • Treat low-precision saved-for-backward activations as precision barriers so forward math observes the same truncated value backward will load (#184110)
  • Fix int to float8 casts in Inductor Triton codegen (#184115)
  • Fix Inductor random fallback for RReLU (#184136)
  • Fix scalar tensor lowering for torch.jagged layout to match eager and meta behavior (#184146)
  • Preserve out_dtype for scaled_mm by propagating it to MMKernelInputs (#184168)
  • Fix Inductor CUDA frexp exponent for non-finite inputs (#184176)
  • Fix Inductor cache directory fallback for unmapped users (#184208)
  • Guard dilation in convolution_backward lowering so dilated conv2d backward compiles under dynamic shapes (#184224)
  • Force persistent reduction for sort in combo kernels (#184227)
  • Fix as_strided storage offsets for graph input views (#184232)
  • Guard dilation to ints in convolution_backward lowering so the Triton backward conv template no longer references undefined sympy symbols under dynamic shapes (#184255)
  • Fix reinplacing through view index_put (#184263)
  • Skip CUDA graphs for aten.topk on ROCm/HIP to avoid a rocPRIM memory fault on cudagraph-tree replays (MI350 / ROCm 7) (#184265)
  • Fix module-table exhaustion during long exhaustive autotuning (#184285)
  • Fix Inductor gather bounds for negative indices (#184287)
  • Fix CUDA graph trees when used from Python-created threads (#184357)
  • Fix Inductor compile pools hanging when running inside daemonic multiprocessing workers (#184472)
  • Fix deterministic selection to correctly detect bound extern (ExternKernelCaller) choices (#184493)
  • Fix CPU integer floor-division overflow so INT_MIN / -1 matches eager instead of trapping (#184497)
  • Include default dtype in the Inductor FX cache key (#184500)
  • Fix Triton compilation failure for scan/cumulative reductions (cumsum/cummax/cummin/cumprod) on unsigned integer dtypes (#184514)
  • Fix Inductor fallback aliasing for boxed scalars (#184516)
  • Extend convolution input/weight dtype-mismatch checks to all devices (#184518)
  • Fall back to the standard lowering path for CUDA FP8 dtypes that Triton cannot compile for the active target (#184521)
  • Check constant_offset alignment in TMACompatibilityChecker (#184564)
  • Fix acc_type for fp8 dtypes in mm autotuning (use triton_type and accumulate in fp32) (#184591)
  • Fix torch.cond subgraph buffer reuse by scoping EfficientPeakEstimate per subgraph during AOT codegen (#184623)
  • Fix backward (gradient) support for addcmul_ and addcdiv_ under torch.compile (#184629)
  • Fix Inductor loop indexing expression canonicalization (#184650)
  • Fix ZeroTensor view with symbolic sizes (#184651)
  • Fix invalid assert-message concatenation in CPU cpp_micro_gemm that broke AOTI compilation of AVX512 VNNI/WoQ-Int4 kernels (#184693)
  • Refuse TMA for known-unaligned buffers and fall back to regular loads/stores (#184717)
  • Record IR reads nested inside constant_args and kwargs in ExternKernel.get_read_writes() to prevent index producers from being incorrectly eliminated (#184751)
  • Map float8 types correctly between torch and Triton for Triton template ops to unblock fp8 max-autotune (#184806)
  • Fix Py_None reference counting in the C++ wrapper (#184869)
  • Emit embedded Triton code and cubin files when loading saved cache artifacts so kernels are not needlessly recompiled (#184953)
  • Fix Inductor subprocess compile pool leaking named semaphore resources when torch.compile runs inside a multiprocessing child process (#185070)
  • Fix TorchInductor CPU codegen crash during vectorization tiling when an index expression contains relational sub-expressions (e.g. torch.compile(dynamic=True) with torch.func.grad) (#185080)
  • Skip TMA codegen for dtypes lacking a tensor-map enum entry instead of crashing at runtime (#185223)
  • Synchronize non-blocking device-to-host copies before a CPU read to avoid a race condition (#185252)
  • Fix AOTInductor C++ codegen most-vexing-parse failure when a cached output is initialized from a constructor-call expression (#185257)
  • Fix CuTeDSL grouped GEMM 'Invalid leading dimension' failure by using get_stride_order for assorted A/B layouts (#185437)
  • Fix a NameError in combo kernels when TMA is enabled by handling DelayReplaceLine load expressions (#185514)
  • Make MSVC compiler detection on Windows robust against non-zero exit codes and empty output to avoid crashes (#185523)
  • Fix a cat lowering assertion error with 1-D statically-empty tensors (#185549)
  • Accept interpreted Triton kernels in wrap_triton when TRITON_INTERPRET=1 is set (#185597)
  • Fix wait_stream reordering by registering it among the synchronization ops that preserve control dependencies (#185627)
  • Fix standalone_compile cache artifact save for grad-enabled graphs whose backward is lowered lazily (#185635)
  • Preserve explicit lowp-fp (fp16/bf16) round-trip rounding on CPU under emulate_precision_casts to match eager (#185847)
  • Separate value-producing symbolic expressions from indexing so codegen uses the correct dtype width (e.g. int64 arange) in tensor value computation (#185853)
  • Gate the narrowing-cast bias-add unfuse on XPU to fix an accuracy regression on ROCm (#185856)
  • Apply a layout constraint to the view.dtype lowering when falling back (#185879)
  • Fix pickling of custom decomposition tables that broke FX graph caching (#185909)
  • Fix FakeTensorUpdater handling of higher-order ops and their subgraphs (#185962)
  • Implement get_stride for PermuteView to avoid a NotImplementedError (#185992)
  • Preserve NaNs for CPU libm calls by avoiding fast-math rewrites in generated C++ (#186018)
  • Fix Inductor crash when logging compute estimations that are SymFloat under dynamic shapes in overlap scheduling (#186054)
  • Fix fake dtype inference for weighted torch.bincount so downstream codegen uses the correct dtype (#186077)
  • Preserve tensor dtype in the scatter_upon_const_tensor rewrite for low-precision const tensors (#186481)
  • Fix Inductor Triton compile/runtime crash from codegen emitting tl.broadcast_to(False, ...) by using tl.full instead (#186621)
  • Fix diagonal_scatter backward under torch.compile (#185146)
  • Fix a SymInt crash in the overlap scheduler's collective/compute node benchmarking (#186065)
  • Allow generator placeholders through control deps (#183863)
  • Fix torch.cat axis handling in Inductor pre-grad fusion (#183995)
  • Fix stale backed-symbol references in AOTI deferred runtime asserts (#184624)
  • Fix AOT FXIR parallel Triton kernel reload (#185134)
  • Fix fp32->bf16->fp32 casts being dropped (#180575)
  • Fix CUDA atan numerics in Inductor (#183984)
  • Fix flip on 0-d tensors in the prims.rev lowering (#184104)
  • Fix the softmax decomposition for symbolic empty dims (#184454)
Ahead-Of-Time Inductor (AOTI)
  • Fix undefined identifier error in CppWrapper due to false-positive caching (#178147)
  • Scale lazy TMA scratch by grid in cpp_wrapper (#182825)
  • Fix folded constant offset indexing in AOTI constant buffer update (#179225)
  • Add GPU stream synchronization after constant folding in AOTI (#181945)
  • Fix use-after-free in pointer_to_optional_list (#183764)
  • Fix Windows AOTI self-mmap size seek (#186386)
  • Promote scalar literals to tensors for AOTI eager compilation (#185313)
  • Use c10::make_scope_exit to avoid exception leaks (#184520)
  • Fix deadlock in AOTInductorModelContainer::run() during concurrent constant folding (#181941)
  • Track and unload CUmodule handles to prevent GPU code object leaks (#184860)
  • Fix MSVC const pointer emission in cpp_wrapper temporary arrays (#179846)
  • Fix MSVC path append in kernel context stack compression (#179857)
  • Add explicit headers for cpp_wrapper to fix MSVC compilation (#180120)
  • Fix inductor AOTI codegen for float('inf')/float('-inf') kernel args (#180297)
  • Fix cond subgraph arrayref dispatch with generic lambda (#180558)
  • Fix arrayref proxy executor tensor args (#182751)
  • Defer Triton compile kickoff out of static init (#182824)
  • Fix cpp_wrapper while loop carried mutations (#183657)
  • Fix AOTI CUDA device copy allocation (#185634)
  • Resolve relative TORCHINDUCTOR_CACHE_DIR (#185723)
Export
  • Fix NaN float scalar input handling in export guard codegen and input constraint checks (#180399)
  • Fix IndexError during decomposition by also excluding lifted tensor constants and custom objects when identifying user-input placeholders (#181179)
  • Fix serialization of predispatch wrapper functions (JVP, vmap) in torch.export save/load (#181263)
  • Fix Triton HOP argument packing to preserve kernel argument metadata (#182101)
  • Simplify Min/Max of scaled symbolic terms (e.g. Min(128*s, 512*s) reduces to 128*s) so export no longer rejects valid branch guards (#185092)
  • Clean up the buffer-registration hook on non-strict export trace failures to avoid a stray AssertionError on later eager buffer assignment (#184956)
  • Fix export dynamic shapes for sparse COO inputs (#184993)
  • Handle scalar tensor slice bounds in non-strict export (#184925)
  • Fix torch.export.load GIL contention during tensor deserialization (#175983)
AOTAutograd
  • Fix torch.compile crash with batched matmul in inference_mode (#181913)
  • Fix expanded output tangent stride handling (#184519)
  • Fix AOT synthetic-base out view returns (#185029)
  • Preserve linalg error checks in AOT graphs (#184111)
Composability
  • Fix data-dependent errors (unbacked SymInts) in pixel_shuffle, pdist, and reflection/replication padding ops (#183814)
  • Add differentiable decomposition for max_pool2d/3d_with_indices (#179104)
  • Add validation for invalid MaxUnpool output sizes in meta/decomposition kernels (#184706)
  • Add validation for invalid conv2d kernel size in meta and symbolic-shape kernels (#180448)
  • Add addmv decomposition dtype validation (#184140)
  • Add fill_ meta value-tensor dimensionality validation (#179363)
  • Add _weight_int8pack_mm meta inner-dims and scales validation (#179364)
  • Fix torch.empty(..., out=...) shape validation under torch.compile (#182349)
  • Fix torch.compile wrong output shape for norm() with a negative dim (#182405)
  • Fix frac decomposition signed-zero handling (#183640)
  • Fix pad_sequence mixed-dtype padding decomposition (#184173)
  • Fix istft fake tensor length padding (#184532)
  • Fix unfold_backward decomposition for overlapping windows (#183996)
  • Fix split Tensor decomposition in Inductor (#184134)
  • Fix embedding negative indices in Inductor (#184107)
  • Fix flash SDPA activation dtype mismatch between meta and CPU implementations (#185573)
  • Preserve 5D nearest upsample decomposition layout (#184553)
  • Preserve aten.hardtanh meta semantics for export (#185298)
  • Fix _fused_dropout decomposition at keep-probability zero (#184979)
  • Fix addmm decomposition crash with out_dtype under FakeTensorMode (#179634)
  • Fix torch.split decomposition for empty dim with nonzero split_size (#181493)
  • Fix torch.distributions.Gamma under torch.compile (#174090)
  • Fix miopen_batch_norm meta save_mean/save_var dtype (#179365)
  • Fix reflection/replication pad stride mismatch under torch.compile (#179837)
  • Fix symbolic float lp_pool2d compilation (#184000)
  • Compare in opmath in hardtanh_backward decomposition (#185840)
  • Use torch.sigmoid() in silu_backward decomposition (#185041)
  • Fix index_copy decomposition shape checks (#184338)
  • Fix LSTM export hidden state metadata (#185716)
  • Fix private convolution fake symint handling (#185081)
  • Preserve strides in meta zero (#185360)
  • Fix runtime check for non_overlapping_and_dense (#186785)
  • Update _cslt_sparse_mm meta registration for hipSPARSELt (#181609)
  • Fix reflection/replication pad output memory format to match eager behavior on XPU (#184484)
  • Handle unbacked dims in folded matmul under FakeTensor (#183397)
  • Preserve unbacked batch dims in SDPA tracing under ProxyTensor (#183398)
  • Fix mix_order_reduction over-fusion via load count check (#179494)
  • Fix torch.compile crash from aten.lift functionalization on an already-functionalized tensor (e.g. randint followed by lift) (#185805)
Quantization
  • Fix a segmentation fault when running fp8 qlinear on x86 CPU without AMX, caused by the qlinear primitive cache (#184317)
Foreach
  • Fix _foreach_sub under compile (#184421)
ONNX
  • Remove incorrect CastLike handling logic from OpRecorder (#182197)
  • Fix _rotary_embedding_23_fake_impl stride drift for 3D and 4D inputs (#184854)
  • Fix invoke_subgraph export with lifted tensor constants (#182230)
  • Fix ONNX dynamic-shape RNN capture (#184872)
  • Fix ONNX type promotion overload selection (#185005)
C++ Frontend
  • Fix crash with invalid embedding bag mode (#186428)
Build Frontend
  • Fix -Winconsistent-dllimport warning in tensor_numpy and tensor_new headers on Windows (#183703)
  • Fix missing symbol exports for ValueError/NotImplementedError on Windows (#175340)
CUDA
  • Workaround for nvrtcCompileProgram changing locale in CUDA < 12.6.2 (#180569)
  • Zero total_weight before accumulating in nll_loss2d (#182082)
  • Fix dtype promotion in max/min kernel (#181505)
  • Round per-process memory fraction cap to avoid spurious OOM (#179444)
  • Fix torch.cuda.ExternalStream(0) to wrap the NULL stream (#183258)
  • Fix stream pool collision in conditional graph nodes (#185836)
  • Preserve internal precision for native_group_norm in eager (#183946)
cuDNN
  • Don't route to cuDNN SDPA for batch size or head dim > 65536 (#180718)
MPS
  • Fix LSTM train/eval error (#180873) and LSTM dropout not being applied correctly (#185351)
  • Fix multinomial SIGSEGV (#180493)
  • Fix async copy failing (#181017)
  • Work around a MetalPerformancePrimitives bug for F.linear on M5+ (#181466)
  • Fix sliced sum reduction (#182688) and Metal unary operator behavior on large strided tensors (#183447)
  • Fix uint32 offset overflow in scatter/gather kernels for strided views crossing 2^32 elements (#182054) and move col2im offset/stride to long to avoid overflow corruption (#185664)
  • Fix NaN handling: relu (#183571), softshrink (#183710), hardsigmoid (#183939), cholesky (#184588), and fast::tanh overflow (#186286); return NaN for std/var on empty input (#184510)
  • Fix layer_norm_backward silent correctness bug for frozen inputs (#183893)
  • Fix Welford reduction codegen with dynamic shapes (#184206), a missing barrier in Welford reductions (#184328), and an Inductor undeclared identifier in multi-pass Welford reductions (#184502)
  • Fix SDPA vector kernel mask offset for partially broadcast masks (#184180) and additive mask scaling in prefill attention (#184400)
  • Fix _amp_foreach_non_finite_check_and_unscale_ zeroing fp16/bf16 grads (#184286) and stop ignoring grad scale and found_inf (#186360)
  • Materialize neg bit in copy_kernel_mps (#184403)
  • Fix sort returning out-of-bounds indices for bool/int-max/NaN inputs (#184620)
  • Fix generator clone (#185002)
  • Fix fill_ on byte-dtype views with misaligned storage offset (#183790)
  • Fix complex exp family on the real axis using precise::sincos (#184749)
  • Make deviceCount() consistent with Python to fix at::manual_seed() (#164571)
  • Fix scale not being cached (#184122)
  • Fix attention compilation on nightly (#186399)
  • Fix the FFT warning (#183061)
  • Disallow bitwise shifts for bool dtype (#186558)
  • Fix bucketization speed/correctness (#185622)
  • Make index copy fast (#185750)
  • Enable and fix large-tensor tests on MPS (#182863)
ROCm

  • Support TheRock wheel distribution in _find_rocm_home (#180723)

  • Fix warpMergeSortTopK padding sentinel for integer dtypes (#182212)

  • Guard ck_group_gemm on USE_ROCM_CK_GEMM (#182615)

  • Fix large arange launch (#182657)

  • Fix triu/tril for 64-bit indexing for large matrices (#179717)

  • Drop dead CUDA/ROCm version gates from tests and helpers (#184879)

  • Fix LayerNorm backward kernel for AMD Strix Halo GPUs (#183864)

  • Decline CuteDSL scatter_add on ROCm (#185678)

  • For HSTU, fix CK flash-attn GQA seqlen_q==1 garbage output (#186434)

  • Inductor fixes:

    • Add config flag to disable pointer_range_32 optimization (#179604)
    • Fix maybe_hipify_code_wrapper for bare-token inputs (#183725)
    • Work around file handle limits in StaticCudaLauncher (#183926)
    • Preserve combo kernel HIP compile options (#180277)
    • lookup_device_info is now case-insensitive (#182284)

  • Windows

    • Fix MIOpen CTC loss crash on Windows (#179264)
    • Apply per-config HIP optimization flags via CMAKE_HIP_FLAGS (#183856)
    • Fix inconsistent dllimport (#183690, #183324, #183282, #183694)
    • Remove redundant cuSPARSE/hipSPARSE error-string forward declarations (#180327)
    • Remove MSVC flags from CMAKE_HIP_FLAGS (#183365)
    • Don't set USE_ROCM_CK_SDPA on Windows (#183962)
XPU
  • Avoid generating fp64 Triton code for XPU devices that do not support fp64 (#180854)
  • Fix stream selection for XPU outputs in CurrentWorkStream (#179140)
  • Fix reflection and replication padding on XPU to preserve eager-mode output memory format (#184484)
  • Fix addmm shape handling and addmv_out stride preservation on XPU (#180985, #178498)
  • Fix XPU deallocation handling and XPUPluggableAllocator registration (#183865, #179392)
  • Fix numerical instability in logcumsumexp with complex inputs on XPU (#174492)
  • Fix SyclExtension Windows builds for oneAPI 2025.3 and later (#170701)
  • Fix getGlobalIdxFromDevice(-1) handling on XPU (#181361)
Benchmark
  • Fix import-time device loading in benchmark timer (#181716)
TorchScript
  • Fix OOB read in MemoryReadAdapter::read (#181193)
  • Validate tensor sizes/strides/storage_offset in C++ Unpickler (#183381)
  • Fix integer overflow in Unpickler storage size computation (#181310)
  • Fix broadcast_shapes op missing in selective builds (#180860)
  • Fix binary_cross_entropy SymInt error with dynamic shapes by registering aten::broadcast_shapes as a TorchScript builtin (#180583)
  • Fix use-after-free in symbolic-shape runtime fusion guard (#183760)
  • Apply bugfixes when enabling Link-Time Optimizations (#180868)

Performance

Autograd
  • Use indexed storage for selective activation checkpointing (SAC) to avoid calling policy_fn during recompute (#176455)
Distributed

  • Speed up store-based metadata exchange on TCPStore by using multiGet and a server-side barrier, reducing network round trips from 2*(world_size-1) to 1 (#182132)

  • Coalesce the NCCL buffer and signal pad into a single symmetric-memory allocation so window registration runs only once (#183344)

  • Fuse slice-cat TP collective patterns (#184911)

FX
  • Skip GraphModule reconstruction in CSEPass when no common subexpressions were eliminated (#185479)
Dynamo
  • Fast path guardless cache hits (#184683)
  • Optimize jagged NestedTensor compile guards (#184053)
  • Skip Dynamo graph break for scalar-only binary ops when tensorify is enabled (#183584)
  • Avoid repr in Dynamo ID_MATCH guard text (#184796)
  • Add a pinned memory pool for activation-offloading ao::offload ops to avoid per-tensor cudaHostAlloc overhead (gated by the pinned_memory_pool() context manager) (#186162)
Inductor
  • Eliminate unnecessary clones for dtype views in auto_functionalize, improving FP8 KV-cache performance (#173177)
  • Fix basic_gnn_sage fp32 single-thread performance regression (#177958)
  • Add a tail-size heuristic for CPP tail-loop vectorization to avoid masking/cast overhead in unprofitable cases (#178243)
  • Raise the split-reduction threshold to 524K on Blackwell+ to avoid over-splitting moderate reductions like softmax/entropy (#179729)
  • Add a pass to replace NVLink collectives with CopyEngine collectives during overlap to remove SM contention with matmuls (#179937)
  • Recalibrate the NCCL analytical model for comm/compute-overlap bucketing with auto-detected bandwidths and per-GPU-generation parameters, improving overlap (up to +14% MFU) (#180463)
  • Add _FastCudaLauncher, a vectorcall C extension for pre-bound kernel launch that reduces per-launch overhead (#180507)
  • Enable PadMM AutoHeuristics by default in deterministic mode, improving deterministic-mode inference speed (~14% on HuggingFace) (#181038)
  • Skip catastrophically slow polynomial sympy.gcd on very wide shape expressions, cutting some backward compiles from over 50 minutes to about 6 (#181275)
  • Add a 128x256x64 (num_stages=4) matmul config that speeds up large Hopper matmul shapes by ~1.3x and up (#181413)
  • Run Inductor joint-graph passes on nested_compile_region (invoke_subgraph) subgraphs created with options, which were previously skipped (#181834)
  • Use scaled persistent configs for the Blackwell scaled-mm TMA autotuning heuristic (#182009)
  • Generate vectorized loads in score-mods for speedups on contiguous, aligned key/value-dimension loads (#183406)
  • Enable reduction-broadcast vertical fusion for block-wise quantization (e.g. MXFP8/MXFP4) by reindexing pointwise nodes before can_fuse_vertical, reducing kernel count (#183521)
  • Use int32 indices in grid_sampler_2d lowering on CUDA/XPU when sizes fit, avoiding unnecessary int64 arithmetic (#184269)
  • Use the two-step variance path for small non-split CUDA reductions for better performance (#184383)
  • Emit evict_first for coalesced last-use loads in persistent reductions (#184395)
  • Use CPU-specific realization heuristic defaults to keep moderate pointwise bodies fused (#184411)
  • Faster FlexAttention CuteDSL codegen via vectorized mask-mod loads and packed-interval span lowering (#184438)
  • Avoid recomputing log in reused CPU pointwise so T5-style softmax bias inputs are materialized once (#184473)
  • Use an Inductor-owned CUDA benchmarking path with an L2-sized cache buffer to avoid Triton's fixed 256MB autotune cache allocation (#184479)
  • Improve compute/collective overlap scheduling by prioritizing collective prefetch candidates that fill more of the available overlap window (#185186)
  • Fuse the decomposed _safe_softmax SDPA math path back into a native scaled_dot_product_attention call (new SFDP patterns 29/30) (#185574)
  • Support CUTLASS EVT epilogue fusion through view/reshape between a GEMM and its pointwise consumer (#185796)
  • Restore dense MKL-DNN pointwise convolution speed by keeping the forward_inference prop kind only for channels-last/MKLDNN-layout, fixing a ~2x dense-contiguous slowdown (#185997)
  • Pre-bucket FSDP collectives in the compile overlap scheduler, merging many per-parameter all-gathers into bandwidth-saturating buckets (#179935)
  • Decompose small dot-shaped batched matmuls (bmm) (#183911)
Ahead-Of-Time Inductor (AOTI)
  • Parallelize tensor-to-bytes conversion for AOTI weight serialization (#181280)
  • Enable shared model loading from a directory to avoid redundant unzipping (#172436)
Composability
  • Use torch.var_mean to fuse paired var/mean reductions (#184843)
  • Avoid a Triton sort compile-time cliff in create_block_mask (#182745)
C++ Frontend
  • Fix reduced-precision rsqrt() double promotion (#181232)
Release Engineering
  • Add an operator microbenchmark comparison workflow for PRs (#179476)
  • Add a batch-invariant accuracy mode for benchmark perf tests (#180610)
CUDA
  • Fix CUDA version check gating warp merge sort (#183527)
  • Allow specifying nbits to radix sort in embedding_dense_backward_cuda (#183578) (#183578)
  • Vectorize scatter_add with TMA bulk reduce on sm_90+ (Hopper) (#182675)
CPU (x86)
  • Speed up random number generation for bfloat16 and half tensors on x86 (roughly 2x faster) by using AVX2/VSX vectorized instructions in the normal_/randn kernel (#179834)
CPU (AArch64)
  • Improve bfloat16 transpose performance on AArch64 by adding a dedicated SVE-vectorized transpose_mxn implementation (#174097)
  • Add SVE128 dispatch for Arm CPUs (#176256)
MPS
  • Fully utilize Philox state in distribution kernels (#182247)
  • 2D dispatch for strided unary kernels (#185291)
  • Templatize Im2Col to regain performance when 32-bit indexing suffices (#185860)
  • Faster norms (#186076)
  • Flatten 5D tensors to 4D in batch_norm for performance (#180335)
ROCm
  • Set MIOPEN_FIND_MODE=FAST in op benchmark CI to prevent cold-cache timeout (#179795)
  • Fix FlexAttention fp16 default num_warps (8 -> 4) on AMD GPUs (#180720)
  • Fix perf regression in index_add and index_reduce (#182533)
  • Add no-fence optimization to the JIT reduce template (#176812)
  • Add target-dependent FlexAttention default forward configs (#181283)
XPU
  • Add oneDNN-backed nn.LSTM inference support on XPU, replacing the per-timestep fused-cell path with a sequence-level primitive (#185531)
Sparse Frontend
  • Optimize the layernorm + sigmoid epilogue by providing an ideal input shape to layernorm after sparse matmul (#183472)

Documentation

Python Frontend
  • Fix out_dtype signatures for bmm, mm, addmm, baddbmm (#179182)
  • Add CUDA SDPA determinism section to randomness.rst (#182551)
  • Convert stub docs pages to MyST Markdown (#183498)
  • Expose nonzero_static docs (#185674)
  • Note expand materialization costs (#185400)
  • Fix torch.trapz documentation signature to match torch.trapezoid (#180571)
  • Clarify that torch.normal does not support integer dtypes (#180580)
  • Document actual keyword argument names for tensor_split (#182075)
Dataloader Frontend
  • Document previously undocumented functions in the torch.utils.data documentation (#182682)
Optimizer
  • Use \gamma consistently for lr in Adafactor math blocks (#184773)
  • Clarify ReduceLROnPlateau threshold_mode behavior (#180638)
Distributed

  • Improve the wording of the batch_isend_irecv documentation (#183022)

  • Add documentation for 8 functions in distributed.md (#182544)

  • Add TorchComms backend documentation to torch.distributed (#182711)

  • Add a distributed training integration guide for out-of-tree accelerators (#182308)

  • Convert the torchelastic elastic/quickstart.rst from reStructuredText to MyST Markdown (#182569)

  • Convert the RPC rpc/rref.rst from reStructuredText to MyST Markdown (#182877)

  • Clarify that --node-rank is only used with static rendezvous (#182374)

  • Add API reference documentation for previously-undocumented functions in torch.distributed.rpc (#183393)

  • Add API reference documentation for previously-undocumented functional optimizer APIs in torch.distributed.optim (#182871)

Distributed Checkpointing
  • Document previously undocumented functions in the torch.distributed.checkpoint API reference (#182887)
DTensor
  • Clarify default dtype behavior in the DTensor.redistribute docstring (#181671)
  • Document undocumented functions in distributed.tensor.parallel.md (#182876)
Distributed FSDP2
  • Document previously undocumented functions in distributed.fsdp.fully_shard (#182866)
Profiler
  • Remove references to _KinetoProfile in public docs (#180672)
Export
  • Fix typos in attention bias, activation, and dataloader docs (#184244)
ONNX
  • Remove stale Caffe2 references from ONNX TorchScript exporter docs (#180498)
C++ Frontend
  • Fix typos in export wrapper docstring and transformer module comment (#181972)

Developers

Optimizer
  • Add optimizer reparameterization helper for non-strict tracing (#181643)
Autograd
  • Expose the PrivateUse1 backend name as an alias in DeviceType (#184835)
  • Show the forward op name instead of the backward node name in autograd anomaly/error messages (#180383)
Distributed
  • Add a missing include to GlooDeviceFactory.cpp (#182800)
  • Fix a missing #include <cuda.h> in CUDASymmetricMemoryTypes.hpp (#183704)
FX
  • Add a fast path in GraphPickler.reducer_override for primitive types (#181602)
Inductor
  • Enable the Inductor SYCL-TLA standalone runner on XPU (developer benchmarking tooling) (#174958)
  • Add an optional Torch-Profiler-based Inductor autotuning benchmarker (off by default, env-gated) for more accurate timing of short-duration kernels (#175097)
  • Add a dedicated custom-pass class for _pre_fusion_custom_pass to match its scheduler-node signature (#179050)
  • Include _post_fusion_custom_pass in the FX graph cache hash (#179051)
  • Share storage for aliased inputs in the generated benchmark_compiled_module repro code to avoid OOM (#181119)
  • Enable frame pointers by default in AOTInductor shared libraries so profilers get accurate stacks (#181358)
  • Fix decomposition and lowering debug info (overload names) for the compiler bisector, including across subprocesses (#181452)
  • Add DOT graph dumps for Inductor debug traces (#184039)
  • Fix the Inductor save_args debug/repro tooling to serialize fake tensors and symbolic metadata so it works with dynamic shapes (#184428)
  • Fix the device argument in the generated combo-kernel benchmark script (#184868)
Ahead-Of-Time Inductor (AOTI)
  • Add C-ABI-safe V2 interface for MinimalArrayref (#179483)
  • Add C-ABI-safe V2 interface for UpdateConstantsMap (#180533)
  • Add C-ABI-safe ExtractConstantsMapForEach (#183030)
  • Add C-ABI-safe UpdateConstantBufferPairs (#183031)
  • Add C-ABI-safe UpdateConstantBufferFromCpuPairs (#183032)
  • Add C-ABI-safe UpdateInactiveConstantBufferPairs (#183033)
  • Add C-ABI-safe AOTInductorModelCreateV2 (#185729)
Composability
  • Change most HOPs to use @register_fake instead of py_impl(FakeTensorMode) (#186247)
  • Genericize graphsafe RNG in aot_autograd to support non-CUDA device backends (#182391)
  • Mark graphsafe_run_with_rng_state as cacheable for FxGraphCache (#185562)
Release Engineering
  • Migrate the build to CMake / scikit-build-core: move NCCL checkout, source-file mirroring, header wrapping, compile_commands merging, and the torch._C extension/version.py build out of setup.py and into CMake (#181450, #177642, #177643, #177644, #180243)
  • Drop the setuptools concat_license_files hook and adopt PEP 639 license-files; replace deprecated distutils usage (#180237, #182120)
  • Install libaotriton_v2.so via cmake install for wheel packaging (#180242)
  • Embed the macOS OpenMP runtime in PostBuildSteps (#180239)
XPU
  • Respect MKLROOT, CMPLR_ROOT, and ONEAPI_ROOT from setvars.sh in FindMKL.cmake so custom oneAPI installs are detected correctly (#183506)
TorchScript
  • Add torch._C._jit_replace_submodule to swap submodules in scripted modules while updating parent types and remapping referenced types across graphs (#180296)
  • Use TORCH_CHECK instead of AT_ASSERT for single input/output node helpers, producing clearer error messages (#181282)
  • Expose overlapsWithUsedNodes and getVmap from SubgraphRewriter (#183333)
View originalPermalink
v2.12.1

PyTorch 2.12.1 Release, bug fix release

This release is meant to fix the following regressions and silent correctness issues:

Regression fixes
  • Fix nondeterministic outputs in test_batch_invariance with FLASH_ATTN on NVIDIA B200 GPUs (#181248), fixed by updating Triton to 3.7.1 (#186814)
  • Fix illegal memory access in the Triton convolution2d_bwd_weight kernel on B100/B200 (sm100) GPUs (#187081), fixed by updating Triton to 3.7.1 (#186814)
  • Fix fill_ on byte-dtype views with misaligned storage offset (#186821)
Releng / Build
  • Drop CPython 3.13t from the binary build matrix (#182951)
View originalPermalink
v2.12.0

PyTorch 2.12.0 Release

PyTorch 2.12.0 Release Notes

Highlights

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Build Frontend

  • Strengthened SVE compile checks in FindARM.cmake, which may reject previously accepted but incorrect SVE configurations (#176646)

    Source builds that enable SVE now validate the compiler configuration more strictly. If a build previously passed with an incomplete or mismatched SVE setup, it may now fail during CMake configuration instead of later in compilation. Update the compiler/toolchain flags so they accurately describe the target SVE support, or disable SVE for that build.

  • Updated the minimum CUDA version required to build PyTorch from source to CUDA 12.6 (#178925)

    Building PyTorch from source with CUDA versions older than 12.6 is no longer supported. Users building custom binaries should install CUDA 12.6 or newer and make sure CUDA_HOME points to that installation.

    Version 2.11:

    CUDA_HOME=/usr/local/cuda-12.4 python setup.py develop

    Version 2.12:

    CUDA_HOME=/usr/local/cuda-12.6 python setup.py develop

  • Enforced a C++20 minimum in CMake build files (#178662)

    Source builds now require a compiler and build configuration that support C++20. If you maintain custom build scripts or downstream extensions that build PyTorch from source, update the compiler and remove assumptions that PyTorch can be built as C++17.

Distributed

  • torch.distributed.nn.functional ops now raise RuntimeError under torch.compile (#177342)

    All ops in torch.distributed.nn.functional (e.g., broadcast, all_reduce, all_gather, reduce_scatter, all_to_all_single) now raise RuntimeError when called inside torch.compile. Users should migrate to the functional collectives API in torch.distributed._functional_collectives.

    Version 2.11:

    @torch.compile
def my_func(x):
    return torch.distributed.nn.functional.all_reduce(x, op=ReduceOp.SUM)

    Version 2.12:

    @torch.compile
def my_func(x):
    return torch.distributed._functional_collectives.all_reduce(x, reduceOp="sum", group=group)
TorchElastic

  • torchrun now defaults to an OS-assigned free port for single-node training instead of port 29500 (#175699)

    When running torchrun --nproc-per-node=N script.py without specifying --master-port or --standalone, the default behavior now automatically uses an OS-assigned free port via the c10d rendezvous backend. This eliminates "Address already in use" errors when running multiple training jobs concurrently. Multi-node training, explicit --master-port, PET_MASTER_PORT env var, and --standalone are unchanged.

    Version 2.11:

    # Used static rendezvous on port 29500 by default
torchrun --nproc-per-node=4 train.py

    Version 2.12:

    # Uses OS-assigned free port by default
torchrun --nproc-per-node=4 train.py

# To explicitly use a fixed port:
torchrun --nproc-per-node=4 --master-port=29500 train.py
MPS

  • All MPS tensors are now allocated in unified memory (#175818)

    Previously, MPS tensors could be allocated in either device-only or unified memory. Now all MPS tensors use unified memory unconditionally. This simplifies memory management and enables CPU access to MPS tensor data without explicit copies. Code that relied on device-only memory placement may observe different performance characteristics.

Inductor

  • The max_autotune layout-constraint deferral introduced in 2.11 is now opt-in (#175330)

    In 2.11, Inductor deferred layout freezing for max_autotune templates to expose more fusion opportunities. This caused a regional-inductor failure mode, so the default in 2.12 reverts to immediate layout freezing. Users who relied on the deferred behavior for fusion opportunities should opt in explicitly via torch._inductor.config.max_autotune_defer_layout_freezing or TORCHINDUCTOR_MAX_AUTOTUNE_DEFER_LAYOUT_FREEZING=1.

    Version 2.11:

    # Deferred layout freezing was the default
torch.compile(model, mode="max-autotune")

    Version 2.12:

    import torch._inductor.config as cfg
cfg.max_autotune_defer_layout_freezing = True
# or set TORCHINDUCTOR_MAX_AUTOTUNE_DEFER_LAYOUT_FREEZING=1
torch.compile(model, mode="max-autotune")

Deprecations

Release Engineering

  • Deprecate CUDA 12.8 builds in favor of CUDA 13.0 (#179072)

    CUDA 12.8 binaries have been removed from the PyTorch binary build matrix. CUDA 13.0 is now the stable default and CUDA 12.6 remains available for users on older drivers. Users explicitly pinning the cu128 index URL will need to switch to cu130 (recommended) or cu126.

    Version 2.11:

    pip install torch --index-url https://download.pytorch.org/whl/cu128

    Version 2.12:

    # Use CUDA 13.0 (default on PyPI):
pip install torch
# Or explicitly:
pip install torch --index-url https://download.pytorch.org/whl/cu130
# Older driver fallback:
pip install torch --index-url https://download.pytorch.org/whl/cu126

  • Compatibility with CMake < 3.10 will be removed in a future release (#166259)

    Source builds against CMake versions older than 3.10 now emit a deprecation warning. A future release will require CMake 3.10 or newer; please upgrade CMake before then.

Linear Algebra

  • Several CUDA linear algebra operators no longer use the MAGMA backend and now dispatch to cuSolver or cuBLAS unconditionally:

    • torch.linalg.eigh now dispatches to cuSolver (#174619)
    • torch.linalg.lu_solve now dispatches to cuSolver/cuBLAS (#174248)
    • torch.linalg.cholesky_inverse now dispatches to cuSolver (#174681)
    • torch.linalg.cholesky_solve now dispatches to cuSolver (#174769)

    User code calling these APIs does not need to change. The practical impact is for users who depended on MAGMA-specific numerical behavior, performance characteristics, or debugging. Those calls now use the cuSolver/cuBLAS implementations on CUDA.

FullyShardedDataParallel2 (FSDP2)

  • Compiling through FSDP2 hooks without graph breaks is no longer supported (#174863, #174906). If you use compiled autograd with FSDP2, update your code to allow graph breaks around FSDP2 hooks or disable compiled autograd for the FSDP2 training step.

    Version 2.11:

    with torch._dynamo.config.patch(compiled_autograd=True):
    compiled_model = torch.compile(fsdp_model, fullgraph=True)
    loss = compiled_model(input).sum()
    loss.backward()

    Version 2.12:

    # Either run FSDP2 backward without fullgraph.
compiled_model = torch.compile(fsdp_model, fullgraph=False)
loss = compiled_model(input).sum()
loss.backward()

# Or apply compile before applying FSDP.
compiled_model_pre_fsdp = torch.compile(model, fullgraph=True)
compiled_model = fully_shard(compiled_model_pre_fsdp, ...)
loss = compiled_model(input).sum()
loss.backward()
Profiler

  • Profiler's metadata_json field is now deprecated; use event_metadata instead (#179417)

    Version 2.11:

    metadata = event.metadata_json

    Version 2.12:

    metadata = event.event_metadata
Dynamo

  • torch.compile(fullgraph=True) now warns when a call runs no compiled code; will error in 2.13 (#181940)

    Previously fullgraph=True was only validated once Dynamo actually compiled and ran the function. If Dynamo was bypassed at call time (e.g. under a user-defined TorchDispatchMode), the annotation silently had no effect. 2.12 emits a warning; 2.13 will raise. For graph-break errors without fullgraph's stronger guarantees, use torch._dynamo.error_on_graph_break.

    Version 2.12:

    from torch.utils._python_dispatch import TorchDispatchMode

class LoggingMode(TorchDispatchMode):
    def __torch_dispatch__(self, func, types, args=(), kwargs=None):
        return func(*args, **(kwargs or {}))

@torch.compile(fullgraph=True)
def model(x):
    return x.sin() + 1

# A user-defined TorchDispatchMode is active, so Dynamo skips the frame
# and no compiled code runs — emits a warning in 2.12, will raise in 2.13.
with LoggingMode(): # Remove this to fix warning
    model(torch.randn(3, 4))

  • The inline_inbuilt_nn_modules Dynamo config is deprecated (#177489, #178205)

    Inlining of in-built nn.Module instances is now the default; setting the flag emits a deprecation warning and it will be removed in a future release.

    Version 2.11:

    import torch._dynamo.config as cfg
cfg.inline_inbuilt_nn_modules = True  # was a tunable knob

    Version 2.12:

    # No action needed — inlining is on by default.
# Remove any explicit references to torch._dynamo.config.inline_inbuilt_nn_modules.

  • Added a deprecation framework to the torch.compile config module so individual options can be marked deprecated (#169837)

New Features

Release Engineering
Python Frontend
  • Introduced torch.accelerator.Graph as a unified frontend Graph interface (#171285)
Foreach
  • Add _foreach_clone operator, with a fast path for CUDA utilizing _foreach_copy_ (#177421)
Distributed
  • Add Store::barrier API and TCPStore client BARRIER support, reducing synchronization round trips compared to the existing ADD+WAIT pattern (#174920)
  • Add NCCL communicator suspend(), resume(), and memory_stats() APIs for managing communicator memory lifecycle (#176300)
  • Add all_to_all support in the Gloo backend (#165435)
  • Add reduce_scatter_offset to symmetric memory, supporting variable-sized block reductions with NVLink multicast or LSA fallback (#177791)
  • Enable batch_isend_irecv to work under torch.compile (#161213)
  • Add torch.distributed.symmetric_memory.is_symm_mem_tensor() API to check if a tensor is a symmetric memory tensor (#178947)
  • Convert NanCheck to a standalone op (torch.ops.c10d.check_for_nan) usable outside of ProcessGroupNCCL (#174990)
DTensor
  • Add support for twice-differentiable DTensor redistribution (#160509)
  • DeviceMesh is now traceable by torch.compile. Make DeviceMesh opaque (#176661), Make placements opaque (#171482).
  • Add grad_placements parameter to DTensor.from_local(), allowing explicit control over gradient placements in the backward pass (#175867)
FullyShardedDataParallel2 (FSDP2)
  • Support per-parameter meshes in FSDP2, enabling different parameter groups to shard over different meshes (#173509)
  • Support fully_shard with DTensors on a full SPMD mesh via DataParallelMeshDims (#176334)
  • Add FSDP2 support for non-floating-point parameters by excluding non-float parameters from reduce-scatter while still sharding and all-gathering them as needed (#177948)
TorchElastic
  • Add configurable --shutdown-timeout to torchrun for controlling the SIGTERM-to-SIGKILL timeout during worker shutdown (#172596)
CPU x86
  • Expose a CPUBlas brgemm API for fp8 (e4m3 & e5m2) GEMM, backed by oneDNN (#172548)
CUDA
  • Added support for torch.cond with CUDA graphs, using conditional graph nodes (CUDA 12.4+) so data-dependent control flow can be captured entirely inside a single CUDA graph. Works with the eager and cudagraphs torch.compile backends (no Inductor support yet). (#168912)
MPS
  • Implemented linalg_qr for MPS (#172536)
  • Added cholesky_solve support on MPS (#176703)
  • Added index_reduce on MPS (#174936)
  • Implemented torch.distributions.Gamma (forward + backward) on MPS (#179228)
  • Enabled mvlgamma on MPS (#178914)
  • Added nonzero_static implementation on MPS (#179589) (from miscategorized)
ROCm
XPU
  • Support torch.accelerator.Graph on XPU (#176421)
  • Added memory_clock_rate and memory_bus_width to XPU device properties (#171967)
  • Enable split_group API when TorchComms is used as a backend for TorchTitan on XPU (#178236)
Profiler
  • Profiler's Activity selection allows for fine-grained activity type selection. (#176351)
  • Memory visualize has a new tab to show private pool memory view (#177289)
Dynamo
  • Made torch._dynamo.aot_compile public, with aot_eager and inductor backend support and docs (#179917, #180008)
  • Added a recompile_limit keyword argument to torch.compile to override the per-function recompile cap without touching global config (#177936)
  • Added min/max bounds to torch._dynamo.mark_unbacked for communicating value ranges to the symbolic shape system (#176313)
  • Added bdb, a pdb-style debugger for stepping through nested frames during Dynamo tracing (n, u, d, r, bt), plus a user-callable breakpoint() that auto-starts it (#174626, #174746, #175200)
Inductor
  • Added user-defined stream support to torch.compile. Inductor now codegens stream context managers (enter/exit) and record_stream calls in the wrapper, enabling user streams to flow through compiled regions with proper synchronization, scheduler integration, and cross-stream dependency tracking (#165390, #165391, #165504, #165505, #174223, #176700, #177694)
  • Added ao::offload, ao::reload, and ao::wait ops for asynchronous activation offloading. These ops encapsulate async CPU offloading stream management following the same async 2-op pattern as c10d functional collectives, reducing IR size from 7 nodes (offload) and 5 nodes (reload) down to 2 nodes each (#177621)
  • Added user-defined Triton kernel unary epilogue fusion. Inductor can now fuse user Triton kernels with downstream pointwise epilogues (e.g. relu()), parsing the user kernel source via AST and inlining the epilogue into the tl.store expression (#173662)
  • Added out-variant discovery and lowering for custom ops. When a custom op registers both functional and .out overloads, Inductor automatically lowers single-output and multi-output functional ops to their .out variants as ExternKernelOut, enabling memory planner buffer reuse (#175116, #176117)
  • max_autotune now extends to combo kernels. The autotuning pipeline generates and benchmarks per-sub-kernel block-size phase configs, with chained sequential autotuning and per-sub-kernel reduction hints (#177715, #178936, #179317)
  • Added non-TMA persistent Triton templates for mm and addmm for max-autotune, enabling persistent kernels on hardware without TMA (#177781, #179095)
  • Added CUTLASS backend support for torch.float8_e5m2 dtype, including registration for FP8 GEMM autotuning (#171176)
  • Added XPU CUTLASS GEMM kernel codegen and codecache to max-autotune-gemm, allowing CUTLASS-style GEMM templates to target Intel GPUs (#161938, #161939)
  • Added MTIA Triton codegen for sort, median, and mode operations (#178525)
  • Added a Triton template for depthwise conv1d (#175280)
  • Added AVX10.2 fp32↔fp8 intrinsics in at::vec::convert for the Inductor C++ x86 backend (#172309)
  • Pallas backend: added scalar prefetch and indirect access support (#177212)
  • Added a disable_welford_reduction config flag to opt out of Welford reduction in codegen (#175778)
Ahead-Of-Time Inductor (AOTI)
  • Add MXFP4 dtype support (float8_e8m0fnu and float4_e2m1fn_x2) to the AOTInductor C shim layer, enabling MXFP4 quantization (e.g., for AMD MI350) (#176496)
  • Add a compile backend registry and custom device support for AOTI Eager, letting out-of-tree device backends plug into the AOTI eager compile/load flow without modifying upstream code (#175605)
torch.fx
  • Add tuple_return option to split_module that wraps submodule outputs in a tuple (#179007)
  • Add ignore_raw_node option to GraphPickler (#176939)
  • Add _merge_overlapping_fusions() method to FxNetSplitter which detects and merges overlapping fusion groups (#177099)
torch.export
  • Add serialization support for float8_e8m0fnu dtype (#176270)
  • Add serialization support for torch.uint32 and torch.uint64 dtypes (#179434)
  • Add serialization support for nested float lists (List[List[float]]) (#178081)
JIT
  • Added input-independent graph optimization API (#179393)

Improvements

Release Engineering
Python Frontend
  • Used compiler wrapper when building C++ extensions (#175696)
  • Updated uniform and normal sampling on CPU to improve fp16/bf16 results (#175988)
  • Changed requires_grad to Optional[bool] in torch.asarray (#170897)
Autograd
  • Implemented narrow_copy derivative (#175609)
  • Implemented higher-order derivatives for grid_sample (#177487)
  • Implemented backward and forward AD for torch.aminmax (#175215)
  • Exposed num_splits in varlen attention to allow disabling split_kv (#176905)
  • Added user AutoNamingMode support in Selective Activation Checkpointing (#175348)
  • Refactored torch.utils.checkpoint to no longer use autograd.Function for saving inputs (#174327)
Dataloader
  • Added a thread-safe RNG utility function (#175375)
Linear Algebra
  • Added _int_mm unsigned int8 × signed int8 (u8s8) support on CPU (#168226)
  • Added FP64 support for TunableOp on ROCm via hipBLASLt (#178195)
Nested Tensor (NJT)
  • Added nested tensor deserialization support (#174843)
torch.nn
  • Added bias argument to nn normalization methods (LayerNorm, GroupNorm, RMSNorm, etc.) (#176573)
  • Improved MultiMarginLoss error message for inconsistent target size (#174072)
  • Added enable_gqa flag to varlen_attn (#179468)
  • Allowed eps=0 in batch_norm during eval mode (#175508)
  • Added meta device support in trunc_normal_ initialization (#176240)
  • Split onehot checks for CPU and accelerators (#181211)
Sparse
  • Implemented clone operator for semi-structured sparse tensors (#174991)
  • Allowed semi-structured sparse tensors to be instantiated with alg_id (#178659)
  • Enabled FP8 semi-structured sparsity on ROCm via hipSPARSELt (#179310)
Build Frontend
  • Simplified SVE256 detection (#176247)
  • Removed ARMv7 checks (#176645)
C++ Frontend
  • Upgraded cpp_extension and cpp_builder to C++20 (#176659)
  • Reland at::Tag header-only changes and add a library.def override for tags (#181608)
Distributed
  • Add configurable worker timeout and partial data support to the distributed debug server (#176058)
  • Add timeout parameter to torch.distributed.barrier() (#174974)
  • Add reduce_scatter_tensor_coalesced support to ProcessGroupWrapper (#168961)
  • Functional collectives API now automatically handles non-contiguous inputs instead of asserting (#177965)
  • DDP: Add batched_grad_copy option to reduce per-parameter kernel launches to 2 kernels per bucket (#176638)
  • DDP: Refactor bucket capacity config into BucketCapacityConfig dataclass (#175217)
  • Add signal name to ChildFailedError exitcode output for better debugging (#175254)
  • Add CUDA-aware detection for Cray MPICH (#178323)
  • Support dist.broadcast for FP8 tensors on GPUs older than SM90 (#175884)
  • Add __torch_function__ handlers for distributed functions (#176376)
  • Enable split_group API for TorchComms on XPU (#178236)
  • Make py-spy dumps nonblocking by default (#178312)
  • Add ncclx and gloo to FlightRecorder trace analyzer backend allowlist (#180268)
  • Improve error message on symmetric memory handle exchange (#178989)
  • SymmMem: Add thread safety to NCCL and NVSHMEM backends (#176551)
  • Check NCCL terminate signal more frequently when exiting from heartbeat monitor (#170000)
  • Implement missing methods in ProcessGroupWrapper (#178779)
  • Add compute_estimator option for overlap scheduling (#175204)
  • [local_tensor] Add standalone rank_map/tensor_map functions (#174795)
Distributed Checkpoint (DCP)
  • DCP: Improve save plan validation error messaging (#176728)
  • DCP: Preserve original exception in metadata read failure for better debuggability (#177739)
DTensor
  • Support DTensor view ops (flatten/unflatten) with _StridedSharding for full nn.Linear(DTensor) compatibility (#166483)
  • Add Dijkstra-based single-dim strategy search for DTensor sharding propagation, avoiding exponential enumeration of strategy combinations (#169438)
  • DTensor: Add is_pinned() support (#177235)
  • DTensor: Add print() HOP support (#175222)
  • DTensor: Emit zero paddings for uneven shardings to enable SPMD compilation (#177758)
  • DTensor: Make run_dtensor_rng_op compatible with compile_on_one_rank (#177447)
  • DTensor: Lenient handling of view redistributes in decomposition flow (#175194)
  • DTensor: Redistribute from/to _StridedShard through Replicate (#179059)
  • DTensor: Raise clearer error for unsupported Split(Flatten) sharding propagation (#179632)
  • DTensor: Unbacked-safe view_groups (#174629)
  • DTensor: Expanded sharding strategy coverage for index_select, index, index_fill, index_reduce, roll, fft, constant_pad_nd, squeeze.dims, interpolate, linalg ops, LayerNorm/RMSNorm FW/BW, foreach/fused ops, and einsum linearity (#176037, #176038, #178456, #175463, #175656, #173563, #176991, #176955, #179173, #177186, #177187, #176150, #174830)
FullyShardedDataParallel (FSDP)
  • Remove mixed-dtype rejection from FSDP clip_grad_norm to match the documented behavior (#173641)
FullyShardedDataParallel2 (FSDP2)
  • Allow ModuleList/ModuleDict subclasses that implement forward() (#175033)
  • FSDP2: Support dataclass args/kwargs output without memory leakage (#174692)
  • Share more implementation code between replicate and FSDP2 fully_shard (#173580)
  • Consolidate FSDP2 shard_mesh and shard_mesh_from_root handling (#174107)
Distributed Pipeline
  • DTensor metadata foundation for Pipeline Parallelism with DTensor-aware stage and schedule refactoring (#177727, #177728)
  • Pipeline Parallel: Dispatch homogeneous P2P ops individually to avoid stream serialization (#175712)
TorchElastic
  • [elastic] Add Windows support for stdout/stderr redirects (#176789)
  • torchelastic: Keep health check alive during exit barrier (#178197)
CUDA
  • [CUDA] Fix offset_t operators to be __host__ __device__ in SortStable.cu (#175997)
  • [CUDA] [Green Context] Add support for workqueue limit (#177242)
  • Remove dead avg_pool3d backward shape-check variables in CUDA (#178893)
  • [BE] add missing assert on cuda device synchronize in ATen tests (#174966)
  • [reland 2][pytorch] Preemptive OOM rejection using per_process_memory_fraction + throw_on_cudamalloc_oom (#179473) (#179473)
  • [cuda graphs] Add enable_annotations kwarg to torch.cuda.graph` (#179867)
  • [CUDA/ROCm] avoid double casting in ReduceLogicKernel (#176132)
  • Nit fix: Align state_step tensor max to param tensor max (#178913)
cuDNN
  • Upgrade 12.8, 13.0 (and 12.9) wheels to cuDNN 9.20.0.48 (#177321)
MPS
  • Fixed abs complex overflow/underflow on MPS (#174346)
  • Migrated index_fill_ to native Metal (#175822)
  • Extended histogram to float/bfloat types on MPS (#176913)
  • Extended unfold_backward to torch.complex64 on MPS (#177274)
  • Added complex input support to scatter, gather, repeat, cumsum, logcumsumexp, cumprod, and nn.functional.linear on MPS (#177794, #178198, #178328, #178411, #178436, #178799)
  • Migrated lerp, eye, relu, silu, fill_, xlogy, norm to native Metal kernels (#177093, #178683, #178866, #179071, #176101, #177749, #177328)
  • Registered DeviceCapability for MPS backend (#178180)
  • Switched exponential distribution to native Metal (#174277)
  • Add enable_gqa parameter to SDPA MPS meta registration (#181550)
ROCm
  • CPP extensions only compile for user's detected arch (#168998)
  • Remove obsolete HIP NaN handling workarounds; remove technical debt (#171104)
XPU
  • Support half precision FFT on XPU backend (#171231)
  • Add proper float64 handling for addmv, addmm, and baddbmm on XPU (#174590)
  • Enable FMA-based addcdiv lowering for XPU (#176163)
  • Enable bmm_outer_product Triton override for XPU (#180816)
  • Use version check for XPU fallback registration in Inductor (#174679)
  • Catch Intel Triton compilation/runtime errors as IntelGPUError in Inductor (#169167)
  • Improve Inductor UT coverage for XPU (#174053, #174054, #174055, #174056, #174057, #174058)
  • Added Uint16/Uint32/Uint64/FP8 support to XPU device capability reporting (#178467)
Profiler
Dynamo
  • Broader Python tracing: enum.Enum iteration, nn.Module.__getattribute__, _enter_autocast/_exit_autocast and other context managers, next() on itertools.count, itertools.takewhile, bool(OrderedDict), NamedTuple.__eq__(tuple), numpy ndarray.flat, and locals()/vars() (#175176, #175527, #173877, #176521, #178818, #177876, #175394, #176729, #175787, #179595)
  • CPython nb_index/nb_bool/nb_float slots so Dynamo can trace operator.index(tensor), bool(...), and float(...); graph-break on torch.Generator methods (#178921, #178931, #179114, #180198, #178519)
  • Higher-order ops & subgraphs: cond supports aliases and mutations under no_grad, autogradable leaf modules support pytree outputs, nonstrict_trace accepts nn.Module inputs, and invoke_subgraph supports subgraph reuse (#172836, #172152, #175010, #172372, #176644)
  • Streams & Triton: current-stream handling via torch.cuda.stream, sync barriers via a dependency HOP, triton.set_allocator inside torch.compile, and reuse of tracked objects for Triton prune_configs_by (#177610, #168894, #177470, #177874)
Inductor
  • Unified OUT_DTYPE, ACC_TYPE, and INDEX_DTYPE codegen flow in Triton templates (#179453)
  • Enabled cudagraph w/o partition for cpp-wrapper (#179249)
  • Added FMA-based addcdiv lowering for CUDA parity with eager and matching _foreach_addcdiv to _foreach_addcmul (#174912, #175309, #175310, #175839, #176237)
  • Added lerp decompositions for bitwise parity with eager (#176804)
  • Added outer-product decomposition (#176552)
  • Enabled padding fusion with torch.cat and avoided duplicate computation in cat/pad when inputs have multiple consumers (#175729)
  • Lowered functional symmetric memory ops to ExternKernelOut for output buffer reuse, and added symm_mem planning for graph inputs and fallback regions (#174856, #175449)
  • Modified addmm template call to support hipblaslt bias-fused kernels on ROCm (#177130)
  • Newly trained PadMM AutoHeuristics for A100 and H200, plus support for pad_mm AutoHeuristics in deterministic mode (#176186, #179826)
  • Propagate metadata in pattern matcher and add validation (#179113)
  • FlexAttention: raise a clear NotImplementedError when return_aux=AuxRequest(max_scores=True) is requested with BACKEND='FLASH' instead of failing later with an opaque error (#177434)
  • Migrated Inductor internals from legacy allow_tf32 to fp32_precision to avoid divergence with the new TF32 API (#176098)
  • Pallas backend: enabled element-wise ops, native TPU OOB DMA masking via aligned block specs, and generalized N-D transpose permutation detection (#174743, #175458, #176952)
  • Registered lowerings for prims.scalar_tensor and aten.arange.start_step (#179017, #179028)
  • Added SDPA pattern matching support for visformer (#177826)
  • Relaxed concat-linear fusion to support GQA QKV (#178523)
  • Allowed subgraphs to be benchmarked with async pipelined autotuning (#175455)
  • Added convert_element_type lowering to emulate PyTorch eager numerics (#176781)
  • Added GEMM configs to XPU autotuning heuristic (#177647)
  • Added kpack Triton compile options on ROCm (#173179)
  • ROCm: enabled exhaustive autotuning for FP8 (#177797)
  • Override decomposition for aten.index_add (#179486)
  • Drop tile_k from nvMatmulHeuristics matching (#176845)
Ahead-Of-Time Inductor (AOTI)
  • Add aten._grouped_mm to AOTInductor fallback ops, enabling cpp-wrapper mode for grouped_mm (#177307)
  • Support lazy Triton kernel compilation for cpp-wrapper on XPU (#179239)
  • Add dynamic shapes support to AOTI Eager via AOTIPythonKernelHolder, allowing a single compiled kernel to serve multiple input shapes (#176018)
  • Support multi-return ops in AOTI Eager (e.g., native_layer_norm, aminmax) (#176019)
  • Allow custom ops with Optional[List[T]] arguments in cpp wrapper (#174460)
  • Add lazy Triton kernel compilation for cpp-wrapper (#175416)
  • Add TMA support for lazy Triton kernel compilation (#175548)
  • Call latest c_shim version for versioned fallback ops (#181548)
  • Add BC-safe c_shim v2 for _scaled_dot_product_attention_math_for_mps enable_gqa (#181549)
torch.fx
  • Update get_source_partitioner to parse nn_module_stack metadata for improved source-based graph partitioning (#175788)
  • split_module now uses _make_graph_module to support lazy recompile (#177907)
  • Fix fuser_utils.topo_sort to produce a stable ordering (#175378)
  • Fix GraphPickler to support nodes with slice() arguments (#175996)
Composability
  • Added DynamicInt __pow__ and __rpow__ methods (#179868)
  • Added scaled_mm_v2 CPU implementation (#176266)

Bug fixes

Release Engineering
  • Fix periodic inductor CI silently skipping all tests (#177695)
  • Fix python docs build hanging in CI (#180177)
  • Avoid installing test dll into Windows wheel and fix libuv copy path (#179024)
  • Fix aarch64 build-osdc using x86 runner on ARC (#179783)
  • Fix stale PYTORCH_RELEASES_CODE_CC dict (#182369)
Python Frontend
  • Fixed torch.isclose broadcast failure with equal_nan=True (#175244)
Autograd
  • Fixed torch.trace backward for non-square matrices (#175068)
  • Added explicit error when layer_norm computes 3rd order derivatives (#176234)
  • Fixed _wrap_sync_node to replace deps in output node's nested args (#178471)
  • Fixed thread-unsafe lazy init of setup_context cache (#179475)
Linear Algebra
  • Fixed addmv backward pass failure (#165777)
  • Fixed determinant gradient for 1×1 matrices (#171225)
  • Fixed linalg.det backward for 0-dimensional inputs (#177498)
  • Fixed cholesky(upper=True) on macOS for matrices larger than block size (#179154)
  • Revert pytorch/pytorch#172340 (#181364)
torch.nn
  • Fixed trunc_normal_ low precision issue when used with half-precision dtypes (#174997)
  • Added dtype check to nll_loss meta function to prevent invalid input types (#175151)
  • Fixed numerical inconsistency in Conv3d.reset_parameters for channels_last format (#175990)
  • Fixed MSELoss failing to compute gradients when inputs have different dtypes (#175743)
  • Fixed GroupNorm backward correctness bug on AMD wavefront-64 GPUs (#178872)
  • Fixed nn.functional.pad compile crash with deterministic mode and replication padding (#177166)
  • Fixed issue #110505 (#176559)
  • Fix FA4 integration for varlen (#177675)
Sparse
  • Fixed torch.bmm(COO, Dense) memory misalignment on CUDA (#175347)
Build Frontend
  • Fixed TORCH_BUILD_VERSION not updating when version.txt changes (#176167)
Distributed
  • Fix _CoalescingManager not passing Opts to allgather_into_tensor_coalesced() (#175379)
  • Fix _CoalescingManager to raise exception when ops in the coalesced list are not the same type (#175573)
  • Fix getenv/setenv race condition causing segfault during NCCL initialization with heartbeat thread (#167523)
  • Fix AsyncCollectiveTensor inputs leaking into compiled regions, causing RuntimeError or silent data corruption in TP + compile workflows (#179849)
  • Fix potential infinite loop in FlightRecorder when multiple ProcessGroups run into barrier (#179449)
  • Fix two forward passes of DDP-wrapped BatchNorm raising error (#175851)
  • Fix USE_RCOM typo to USE_ROCM in intra_node_comm.cpp (#175078)
  • Fix HPU backend mapping issue (#174764)
  • Fix NCCL symmetric memory mismatch by using allocation-time counter for Block ordering (#178362)
  • Fix NCCLPeerAllocInfo destructor to properly deregister windows and free resources (#177459)
  • Fix nested DDP causing _active_ddp_module cleared by inner _inside_ddp_module() (#178364)
  • Fix extra deps mapping and cycles after bucketing (#177688)
  • Add proper skips for FP8 on sm < 89 (#170528)
  • Fix cross type bucketing (#175150)
Distributed Checkpoint (DCP)
  • DCP: Fix save plan caching bug during validation failures (#176289)
  • DCP: Fix Metadata.storage_meta regression from dataclasses.replace() (#178001)
  • DCP: Fix unpicklable FrameSummary._code on Python 3.13+ (#177754)
DTensor
  • Fix DTensor subclass __torch_dispatch__ bypass (#177878)
  • Fix symbolic shape handling by copying symbolic shapes as needed (#178210)
  • Fix _StridedShard not in safe globals for checkpoint loading (#178560)
  • Fix DTensor stack dim normalization (#174640)
  • Fix DTensor view_as_complex with P(max)/P(min) placements (#173935)
  • Fix DTensor get_mesh_from_args when first arg is not a tensor (#169265)
  • Fix DTensor tp_conv rejecting batch-dim-only sharding for valid configs (#176448)
  • Fix DTensor compute_local_stride for unevenly-sharded tensors (#177174)
  • Fix DTensor scaled_mm sharding strategy (#177234)
  • Fix DTensor double-shard validation in propagate_shape_and_sharding (#177973)
  • Fix DTensor Dijkstra sharding search shardability checks and graceful fallback (#177167)
  • Fix DTensor index_put sharding strategy for None indices (#179217)
  • Fix DTensor precision loss in NestedRedistribute backward dtype handling (#179495)
  • Fix DTensor backward for value-selecting reductions (topk, sort, min, etc.) (#178668)
  • Fix DTensor InputDim.__eq__ type guard to prevent int comparison bugs (#178599)
  • Fix None IValue == DTensorSpec, cache key collision, and move op_strategy_context (#178442)
  • Fix DeviceMesh.__getitem__ by disabling proxy tensor handling (#176007)
FullyShardedDataParallel (FSDP)
  • Fix FSDP _unshard() passing a CUDA stream where an event was expected (#170525)
  • Fix symbolic context reuse for gradients when creating meta tensors (#176864)
  • Fix HSDP sync_module_states broadcast order for buffers with meta-device initialization (#178569)
  • Fix activation checkpointing crash when passing BlockMask as an argument (#179215)
  • Revert a FSDP1 all-gather prefetching regression (#181667)
  • Revert PR 178223 to bring back all-gather prefetching (#181669)
FullyShardedDataParallel2 (FSDP2)
  • Fix FSDP2 split_with_sizes_copy() missing dim argument (#169173)
  • Fix mixed DTensor errors with nested FSDP and activation checkpointing (#171779)
  • Fix tied weights with uneven sharding across separate FSDP groups (#176225)
  • Revert FSDP2 communication-op FQN annotations due to async_op=True profiler trace issues (#182100)
  • Revert "[FSDP2] add fqn to communication ops" (#182157)
Distributed Pipeline
  • Fix stage_backward_weight with multi-output intermediate in pipeline parallelism (#175705)
CPU aarch64
  • Add vectorized BF16 transpose_mxn specialization for AArch64 SVE, providing a deterministic vectorized BF16 transpose implementation independent of fixed vector widths (#174097)
CUDA
  • Fix cuda torch.topk index bug for inputs over 32-bit length (#176095)
  • Fix test/inductor/test_fp8.py hang on sm89 (#177573)
  • Use fp8 conversion intrinsics on Hopper+ to work around ptxas codegen bug (#177870)
  • [CUDA] Fix wrong non-atomic handling in AdaptiveMaxPooling2d.cu (#179261)
  • [CUDA] Fix wrong ComplexTransform const kTransformB in fpA_intB_gemm.h (#179271)
  • [CUDA] Fix wrong LayoutB in fpA_intB_gemm.h (#179269)
  • Back out "[CUDA][cuBLASLt] set cuBLASLt as a default BLAS backend when available (#174594)" (#177703) (#177703)
  • Fix CUTLASS illegal memory access via subprocess isolation (#172123)
cuDNN
  • [cuDNN][SDPA] Don't route to cuDNN SDPA when dropout probability is not a multiple of 1/16 (#174245)
  • Fix cuDNN SDPA with zero-stride (broadcast) Q/K/V inputs (#175764)
  • [cuDNN][SDPA] Fix attn-mask conversion constant (#177868)
MPS
  • Fixed AvgPool for channels_last + offset inputs (#175235)
  • Fixed linalg_solve to return pivots (#175284)
  • Fixed lu_solve for broadcasted bias (#175332)
  • Fixed addmm/mm to return zero-filled matrix when an input is empty (#175905)
  • Fixed index_reduce atomic misalignment for sub-32-bit types (#176009)
  • Fixed masked_fill for non-contiguous outputs (#176171)
  • Fixed layer_norm with noncontiguous bias (#176238)
  • Added unsigned int types to Metal cast operations (#176343)
  • Fixed solve_triangular for noncontiguous inputs (#176335)
  • Fixed histogram/histogramdd with noncontiguous weight (#175906)
  • Fixed MPS memory leak in getStridedMPSNDArray (#176648)
  • Added error checking for bmm on MPS (#176771)
  • Fixed half-precision type mismatches in Metal shader codegen (#176436)
  • Fixed SDPA output shape when value head dim differs (#176843)
  • Added error when creating torch.cdouble tensor on MPS (#176985)
  • Fixed _copy_from_and_resize logic (#177606)
  • Fixed linear backward crash with channels_last grad (#178278)
  • Fixed mm padding overflow and incorrect alignment conditions (#178203)
  • Fixed nested ops.masked variable name collisions in Metal codegen (#178304)
  • Fixed in-place self.add_(other, alpha) RuntimeErrors with type promotion (#178724)
  • Fixed BatchNorm with mixed input/weight dtypes (#178775)
  • Fixed hi/lo swap typo in Metal Philox RNG (#179227)
  • Allowed getMPSScalar construction for uint64 (#179230)
  • Fixed mm with stride-0 inputs on macOS < 26.4 (#180236)
  • Fixed masked_scatter side-effect and aligned behavior with CPU (#175622)
  • Fixed lgamma/digamma/polygamma noncontiguous behavior (#175603)
  • Fixed masked_scatter to preserve scalar tensor shape (#174381)
  • Fix sliced channels_last tensor handling (#181107)
  • Fix SDPA wrong output for permuted q/k/v with B > 1 (#181886)
  • Fix bool mask handling in the 1-pass SDPA decode kernel (#182311)
ROCm
  • Fix SDPA build error when USE_FLASH_ATTENTION=0 USE_MEM_EFF_ATTENTION=1 (#177552)
  • Fix _get_amdsmi_device_index to return devices in correct order (#178398)
  • Fix scaled_mm incorrectly validating unsupported swizzle (#178688)
  • Move rocblas.h include out of anonymous namespace (#178767)
  • Don't crash for MHA backward with head dim > 192, fall back to CK tile (#178946)
  • Don't fail torch.cuda.device_count() if pynvml is installed (#175077)
  • Fix hipblaslt GEMMs executing concurrently on multiple HIP streams (#179053)
  • Windows
    • Fix linker failure caused by missing DLL export directives via native headers (#179138)
    • Fix int4mm std::memcpy build error (#175410)
    • Fix Windows access violation in MIOpen CTC loss dispatch (#178284)
    • Fix Windows DLL linkage for batch norm (-Winconsistent-dllimport) (#179706)
  • Workaround hipGraph event query errors in NCCL watchdog (#175377)
  • Fix linker error for aotriton when USE_MEM_EFF_ATTENTION=ON but USE_FLASH_ATTENTION=OFF (#175079)
  • Fix build_amd.py (hipify) failure when MSLK submodule is missing (#175180)
  • Hipify CUdeviceptr in lazy scratch allocation codegen (#179978)
XPU
  • Fix torch.compile graph break inside torch.autocast('xpu') causing dtype mismatch (#180309)
  • Fix conv2d incorrect results and alignment errors for non-64-byte-aligned tensors on XPU (#177956)
  • Fix nn.Embedding module failures on XPU (#178987)
  • Fix XPU OneDNN symbol leak (#172437)
  • Fix meta kernel for _scaled_dot_product_fused_attention_overrideable to preserve query layout (#178986)
  • Fix tensorwise scaling settings on XPU (#177810)
  • Fix DeviceOpOverrides registered incorrectly on XPU (#178959)
  • Fix SyclExtension Windows build for oneAPI 2025.3+ breaking change (#170701)
Dynamo
  • Guard correctness: missing source annotation on float guard after recompile, missing guards on class attribute access for literals/enums, guard on constant function __defaults__, guard tensor-method fallthrough against unknown methods, and closure-hash in CodeId so factory functions don't reuse stale graphs (#177103, #177191, #178420, #177737, #173512)
  • Tracing: @property setters bypassed by torch.compile, AttributeError swallowed by try/except on tensor attributes, torch.Size dict lookups with tensor-backed keys, graph break on enum members with class values, detach_ autograd metadata, allow_in_graph crash inside compiled functions, preserve original exception in GuardOnDataDependentSymNode, and einops 0.6.1 backwards patch (#176624, #175611, #177313, #177439, #177875, #178524, #176016, #177165)
  • Nested graph breaks: global-scope bug in nested closures, decorators in the compiled region, graph break in contextlib.contextmanager init, and parent-stack corruption in step_graph_break (#176906, #177090, #177195, #177408)
  • Compile-state & integration: torch.compiler.is_exporting() returning True during torch.compile, activation-checkpoint metadata loss through custom autograd.Function, mixed-dtype bmm/matmul, vjp_fn under torch.compile, _extract_distributed_info crash on FX-Node group_name, nested_compile_region cache keyed on fn.__code__, and reverting allow_in_graph deprecation warn-spam (#176499, #177396, #177696, #173883, #178108, #179148, #178340)
  • Fix cuda_stream pointer extraction for generic torch.Stream (#181019)
  • Warn instead of erroring on fullgraph=True fallback to eager (#181940)
AOTDispatcher
  • Fixed AOTAutograd crash on no_grad views of differentiable intermediates (#175673)
  • Fixed inplace checks in autograd backward functions during functionalization (#177213)
Inductor
  • Fix floordiv Inductor lowering for mixed signedness (Triton workaround) (#175168)
  • Use Sm100CollectiveEpilogue on SM100 (#175305)
  • Fix aten.resize on overlapping-stride views (#176651)
  • Fix ConstructorMoverPass replacing CPU placeholder in graph output and creating mixed-device pointwise ops (#176164, #177646)
  • Use VecMask::from for scalar masks in CPU codegen (#178148)
  • Fix triton_main_loop_scaled_mm template to use correct scale recipe (#178005)
  • Fix cpp_wrapper lazy compile stale state across fresh_cache resets (#178162)
  • Fix int64 indexing with >65k M/N size (#172925)
  • Fix BMM Triton template grid_y overflow for large batch dims and i32 overflow in template kernel signature for large storage offsets (#178617, #179333)
  • Fix remove_no_ops incorrectly eliminating ops on mutated values (#174938)
  • Fix negative-zero constant codegen for the Triton backend (#176035)
  • Fix coordinate descent tuner incorrectly re-running on warm cache (#173391)
  • Fix BF16/FP16 scalar comparison to match eager (#175807)
  • Defensively check name is in buffer_read_counts before access (#171245)
  • Fix cpp-wrapper SyntaxError when Triton kernel has docstrings (#176796)
  • Fix fallback_random dropout stride mismatch (#177077)
  • Fix AssertionError in ForeachKernelSchedulerNode loop reordering after fusion (#176849)
  • Fix incorrect rounding of floordiv (#177926)
  • Fix issue in AMD persistent_mm_template selection (#178178)
  • Pointwise configs with max_autotune must include pointwise configs with max_autotune_pointwise (#177995)
  • Fix num_warps when max_autotune is enabled on HIP (#178023)
  • Fix nn.Dropout accuracy discrepancies between Triton and torch implementations (#178843)
  • Fix eager/compiled mismatch for integer floor_divide with zero divisor (#178016)
  • Accept 1D bias in addmm ATen heuristic on ROCm (#179087)
  • Fix randn_like inconsistency between eager and compile with fallback_random=True (#177994)
  • Fix MetalScheduling constructor in MPSInductor (#179646)
  • Prevent cross-stream inplace and memory-planning buffer reuse for user-streams (#178548, #178549)
  • Fix argmax/argmin returning incorrect indices for boolean tensors on CUDA (#174076)
  • Fix block-pointer advancement for broadcasted tensors (#175008)
  • Fix masked vectorization for the Inductor C++ backend (#174648)
  • Fix bucketize NaN handling in Triton codegen and Pallas sign to match PyTorch NaN semantics (#176579, #176814)
  • Fix UnicodeDecodeError in Triton depthwise conv template (#176484)
  • Handle 0-sized dimensions in Pallas codegen (#176813)
  • Fix torch._check divisibility propagation to Triton tt.divisibility (#175755)
  • Fix SIGSEGV on AMD RDNA (Wave32) from removed reduction masks in persistent kernels (#176269)
  • Pallas: fix non-stride-1 reductions and prevent incompatible reduction fusion (#176489)
  • Fix block_ptr store dtype for inplace-mutated buffers (#177860)
  • Fix constants handling for Triton constexpr (#172354)
  • Fix Inductor reinplace bool shadowing ('bool' object not callable) (#176090)
  • Fix Inductor _split_iteration_ranges silently dropping dimensions (#177673)
  • Fix sym_sum lowering to accept varargs (#178661)
  • Fix bias_addmm for AMD (#178929)
  • Define unbacked slice size symbol when bounds become provable after tracing (#178897)
  • Add isinf() to Float8_e4m3fn to fix nan_asserts crash with fp8 inputs (#160641)
  • Fix division by zero in the Triton kernel launcher when Grid2DWithYZOverflow (#178878)
  • Prevent user-kernel fusion with non-unary epilogues (#179735)
  • Preserve order of torch.cond (#179457)
  • Fix performance regression caused by user kernel epilogue fusion (#176772)
  • Fix torch.compile performance regression for cumprod backward (#170388)
  • Include lazy_triton_compile.h in the XPU cpp_wrapper header (#180815)
  • Fix cudagraphs compatibility with the current stream (#180916)
  • Revert native API stamp-out for BMM outer product (#181658)
  • Fix dynamic shape tile issue (#181795)
  • Avoid raw stream name collisions in Inductor (#182178)
Ahead-Of-Time Inductor (AOTI)
  • Fix AOTI incorrect loads from bool tensor pointers in user-defined Triton kernels (#176353)
  • Fix lazy compile kernel state collisions across modules by making it per-module instead of global (#178163)
  • Fix expression-nesting limit in cpp-wrapper when combo kernel gets too large (#180217)
  • Fix const folding in run_single_threaded (#174998)
  • Fix AOTI Eager caching to populate the in-memory cache after first compilation, avoiding repeated disk round-trips on every dispatch (#176017)
  • Fix SIGPE by adding additional check logics in the codegen (#170669)
  • Fix scratch size for TMA in C++ wrapper (#175385)
  • Emit int64_t type declaration for kernel numel variables (#176922)
  • Fix CPP wrapper lazy compile for scalar tensor args (#178478)
  • Fix Triton kernel stream for user stream contexts (#178547)
torch.fx
  • Fix edge case in get_source_partitioner (#175935)
  • Fix make_fx handling of value types for opaque objects so values are inlined into the graph consistently with dynamo behavior (#178036)
  • Fix meta["val"] not being populated for reconstructed opaque nodes, resolving partitioner classification issues (#179660)
  • Fix repeat_interleave fx graph to be runnable (#177909)
  • Fix fuse_by_partitions crash when partition has no external outputs (#175203)
  • Fix set_stack_trace (#177332)
  • Handle weakref objects during graph serialization in GraphPickler (#178190)
  • Fix horizontal fusion bug and add partition tests for regional inductor (#178421)
torch.export
  • Fix export serialization to handle unused List[Tensor] outputs (#176677)
  • Fix MPS SDPA meta kernel to avoid data-dependent branching (#177620)
  • Fix torch.export.unflatten KeyError when module FQNs contain @N suffixes (#179278)
  • Fix run_decompositions() failure for custom ops with List[List[Tensor]] arguments (#176355)
  • Fix torch.quantile DataDependentOutputException failure (#174859)
  • Fix isin decomposition with using symbolic shapes (#178076)
Quantization
  • Fix activation quantization creating duplicate backward placeholders (#180287)
JIT
  • Silenced CPython 3.13.8 inspect.getsourcelines() bug (#179066)
  • Fixed data race in opaque type registry (#175694)
  • Fixed xplat build: use PyObjectType::get() directly (#178786)
Functorch
  • Fixed grad/vjp/jvp returning zeros under inference_mode (#177596)
  • Fixed vmap batch rules for group_norm backward operator (#176272)
  • [functorch] Fix double-pop in popDynamicLayerStackToDepth (#177585)
torch.func
  • Fixed scatter error messages for inplace operations (#179420)
Composability
  • Fixed stride handling in FFT meta registrations (#175731)
  • Fixed exception messages displaying as tuples instead of formatted strings (#175957)
  • Fixed one_hot runtime error (#177160)
  • Fixed wrong bool-to-int conversion in symbolic tracing (#177178)
  • Preserved scalar item() semantics for size-1 tensors (#177270)
  • Fixed _build_proxy_for_sym_expr for n-ary sympy.Add by mapping to torch.sym_sum (#175398)
FX
  • Fix the MetaProxy error caused by skipping dispatch (#181170)
  • Preserve FakeScriptObject for value-type opaques (#181454)

Performance

Release Engineering
  • Add deterministic mode for benchmark perf tests (#178233)
  • Fix subprocess benchmark crash for addmm with input_reorder (#177930)
  • Add unbacked perf testing to inductor periodic (#177034)
Autograd
  • Used non-blocking copy in save_on_cpu pack hook for faster activation offloading (#175421)
Linear Algebra
  • Improved torch.cholesky_solve performance for batched inputs on CUDA via cuSolver (#175898)
torch.nn
  • Added NEON implementation of interpolate for bilinear/bicubic with antialias on ChannelsLast RGB images on ARM (#176217)
  • Parallelized upsample_bicubic2d across batch/channel dimensions — 4-43x speedup for VLM position embedding resizing (#174578)
  • Freed q, k, v early in multi_head_attention_forward to reduce peak memory usage (#178452)
Foreach
  • Increase kernel argument size 4KB -> 32KB for CUDA 13+ (#178641)
  • Add multi-tensor fast path for p=0 norm (#179869)
Sparse
  • Reduced CPU overhead in sparse operations for improved performance (#179193)
  • Minor performance improvements for torch.bmm(COO, Dense) (#175347)
Distributed
  • Improve AsyncMM.cu performance by avoiding redundant IO/compute via ElementC void type (#178653)
  • Improve Context Parallel head-tail load balancer indices creation performance (up to 1555x speedup for 1M sequence length) (#178199)
  • Improve tensor-to-allocation lookup in NCCL Symmetric Memory (#176744)
  • Avoid two probes when inserting handle into SymmMem cache (#177463)
Distributed Checkpoint (DCP)
  • Optimize DCP consolidation I/O for remote mounts (from ~2h to 35s) (#175762)
DTensor
  • Improve DTensor performance for torch.cat and pytree ops (#174879)
  • Skip unnecessary all-reduce of total_weight in DTensor nll_loss_backward for reduction='sum' (#177233)
  • Cache DecompStrategy and fake mesh in DTensor (#175205)
FullyShardedDataParallel (FSDP)
  • Use SymmMem for reduce-scatter in FSDP (#177111)
FullyShardedDataParallel2 (FSDP2)
  • Improve FSDP2 parameter FQN lookup from quadratic to linear complexity (#174675)
  • Overlap FSDP2 reduce-scatter with compute for per-parameter meshes (#177319)
  • Cache FSDP2 shard_mesh to avoid repeated submesh creation (#179655)
CPU aarch64
  • Add TLS stack_bounds to avoid expensive reads (#181137)
CUDA
  • Update eigh CUDA heuristics (#175403)
  • [CUBLAS][Blackwell] Enable 32MiB cuBLAS workspaces on Blackwell (#175344)
  • [CUDA] [PERFORMANCE] Improve performance for RowwiseScaledMM.cu by avoiding redundant IO/compute via indicating that indicating that ElementC type is void (#178644)
  • [CUDA] [PERFORMANCE] Improve performance for ScaledGroupMM.cu by avoiding redundant IO/compute via indicating that indicating that ElementC type is void (#178325)
  • [CUDA][TensorIterator] Improve vectorized elementwise kernel: instruction cache (#175336)
  • [pt] Reland vec8 vectorization (#176352)
  • Use aminmax instead of min and max kernels in histc (#178011)
MPS
  • Reimplemented cross as single-stage Metal kernel (#175498)
  • Replaced MPSGraph nonzero with native Metal prefix-sum + scatter (#178484)
  • Sped up RMSNorm on MPS (#180173)
  • Removed .item() sync in _amp_non_finite_check_and_unscale_mps (#180267)
ROCm
  • Directly access GPU scalars if largeBar is enabled, avoiding D2H copy (#177023)
  • TopK operator performance improvement via RadixSelect prefetching (#174897, #177149, #178188, #174837)
  • Improved kernel loop unrolling by leveraging compiler (#177697)
  • Remove need for expensive fence in normalization kernel (#175286)
  • Avoid double casting in ReduceLogicKernel (#176132)
  • In group_gemm, use new kernel for all K equal cases (#173502)
  • Use BFloat16 native hardware type casting (#178814)
  • Use optimized tiled kernel for LayerNorm gamma beta backward (#179019)
  • Tune Flex-Attention occupancy on gfx942 for head_dim 64/128/256 (#176261)
XPU
  • Remove unnecessary device-to-host synchronization in torch.nn.functional.one_hot for XPU by skipping boundary validation checks only needed on CPU (#179831)
  • Add GEMM configs to XPU autotuning heuristic (#177647)
Dynamo
  • Faster guards: GUARD_VALUE_DISPATCH dispatch table, fast-path requires_grad, DICT_NOT_CONTAINS/SET_NOT_CONTAINS, PyType_GetDict on Python ≥3.12, thread-safe dict-version tracking, no recompiles on hoistable opaque objects, and a general guard-optimization pass (#176033, #177158, #176053, #179170, #178703, #176643, #175006)
  • Faster tracing: tree_map_with_path, constant-folding elementwise_dtypes, inline_invoke_subgraph post-tracing pass, emit subgraph output intermediates only on observed side effects, drop unnecessary realize_all in speculate_subgraph, skip SymInt copies in TENSOR_SUBCLASS_METADATA_MATCH, and avoid closure refcycle in _empty_create_subclass (#174146, #177743, #176082, #177368, #176742, #175596, #175660)
AOTAutograd
  • Skipped expensive debug_lines computation in AOT autograd cache (#179733)
  • Sped up ConfigModule._get_dict by avoiding unnecessary work (#179734)
Inductor
  • Add donate_graph_module option to standalone_compile to avoid extra graph module copies (#179910)
  • Rewrite multi-consumer F.pad as torch.cat for zero-copy (#177216)
  • ROCm: enable pipelining for FlexAttention (#176676)
  • CPU: make remove_identity in-place for inference to align with pre_grad_passes (#177805)
  • CPU: fuse round and to in quant (#171699)
  • Fix hardcoded OMP thread counts in torch.compile (#170585)
Ahead-Of-Time Inductor (AOTI)
  • Batch cubin-to-obj conversion using .incbin assembly, dramatically reducing AOTI compile time for models with many Triton kernels (e.g., ~640x speedup on the cubin embedding phase for a 4-layer MoE with 661 cubins) (#177864)
  • Parallelize PTX-to-fatbin compilation when emit_multi_arch_kernel is enabled, saving several minutes on AOTI export for large models (#177904)
torch.fx
  • Fix quadratic name generation in _NamespaceBase.create_name, significantly improving performance for graphs with many nodes (#176515, #177217)
  • Propagate custom annotations to runtime asserts (#170796)
torch.export
  • Skip duplicate tensor serialization when saving multiple exported programs (#176880)
ONNX
  • Optimized _jit_pass_onnx_deduplicate_initializers from O(N^2) to O(N) (#175888)
JIT
  • Cached can_compile_class and short-circuited type inference for ProcessGroup (#179396)
Functorch
  • [FUNCTORCH] Use [] instead of list() for improved performance (#175491)
Composability
  • Decomposed mm/addmm to pointwise multiply when K==1, yielding up to 1.55x speedup for outer-product-like matrix multiplications (#175825)
  • Improved tracing speed via the aggressive_guard_free_semantics config flag (#174654)
  • Reduced threshold for calling sympy.factor to 50, avoiding expensive symbolic simplification on large expressions (#177779)
  • Added per-SymNode expression cache, reducing redundant symbolic computation (#175353)

Documentation

Release Engineering
  • Auto-detect missing doc redirects for moved/deleted files (#173805)
  • Add .nojekyll file creation in CPP doc push script (#179721)
  • Simplify condition for linux-docs job (#180391)
  • Skip llms-full.txt during Sphinx builds and generate it in nightly push (#181070)
  • Disable llms-full.txt (#181141)
  • Fix link to C++ torch stable docs (#181613)
  • Use full clone for docs build to fix nightly hang (#181661)
  • Add checkout-mode input to setup-linux action (#181702)
  • Make docs build behave the same for push=true and push=false (#181921)
  • Reduce sidebar navigation size for generated API pages (#181943)
Python Frontend
  • Documented get_device_capability and clarified supported dtypes (#178397)
  • Fixed typo in index_copy doc (#175843)
  • Updated amax doc (#175863)
  • Updated take_along_dim doc (#175844)
torch.nn
  • Fixed varlen attention docstring (#175261)
  • Fixed device mismatch in scaled_dot_product_attention docstring example (#178684)
  • Fixed incorrect attn_mask shape in scaled_dot_product_attention docs (#177999)
  • Clarified RMSNorm eps parameter default behavior (#173887)
  • Improved Conv2d docs: clarified math variable to parameter mapping and fixed cross-correlation link (#178965)
torch.optim
  • Add minimal usage example to Muon optimizer docstring (#177029) (#177262)
C++ Frontend
  • Improved stable ABI docs for header-only dispatch macros and STD_TORCH_CHECK (#177996, #177997)
  • Restructured C++ docs (#174096)
Distributed
  • Document FSDP2 communication grouping and scheduling semantics (#176318)
CUDA
  • [Typo] ot -> to (#179265)
  • [BE] Tesor -> Tensor (#175061)
  • [Typo] Quiet -> Quite (#179266)
  • Document public APIs using a Claude Skill (#175578)
Dynamo
  • Improve nonstrict_trace documentation (#172395)
AOTDispatcher
  • Documented supported input and output dtypes for custom ops (#175452)
Inductor
  • Fix incorrect remediation instructions in cudagraph pending-backward warning (#176865)
torch.fx
  • Update and correct the documentation for Cond operator (#175419)

Developers

Release Engineering
  • Update CXX_STANDARD to C++20 across build targets (#178343)
  • Remove legacy_nvidia_driver code (#175363)
  • Change default CUDA arch list to sm_7.5 (#175574)
  • Move binary build scripts from .circleci/ to .ci/pytorch/ and clean up old copies (#175930, #175915, #175917)
  • Add ARC runner label mapping config and experiment support to runner determinator (#177803, #177804)
  • Use sccache when available for faster builds (#175556)
  • Standardize ninja installation to PyPI (#179508)

…

View originalPermalink
v2.11.0

PyTorch 2.11.0 Release

PyTorch 2.11.0 Release Notes

Highlights

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Release Engineering
Volta (SM 7.0) GPU support removed from CUDA 12.8 and 12.9 binary builds (#172598)

Starting with PyTorch 2.11, the CUDA 12.8 and 12.9 pre-built binaries no longer include support for Volta GPUs (compute capability 7.0, e.g. V100). This change was necessary to enable updating to CuDNN 9.15.1, which is incompatible with Volta.

Users with Volta GPUs who need CUDA 12.8+ should use the CUDA 12.6 builds, which continue to include Volta support. Alternatively, build PyTorch from source with Volta included in TORCH_CUDA_ARCH_LIST.

Version 2.10:

# CUDA 12.8 builds supported Volta (SM 7.0)
pip install torch --index-url https://download.pytorch.org/whl/cu128
# Works on V100

Version 2.11:

# CUDA 12.8 builds no longer support Volta
# For V100 users, use CUDA 12.6 builds instead:
pip install torch --index-url https://download.pytorch.org/whl/cu126
PyPI wheels now ship with CUDA 13.0 instead of CUDA 12.x (#172663, announcement)

Starting with PyTorch 2.11, pip install torch on PyPI installs CUDA 13.0 wheels by default for both Linux x86_64 and Linux aarch64. Previously, PyPI wheels shipped with CUDA 12.x and only Linux x86_64 CUDA wheels were available on PyPI. Users whose systems have only CUDA 12.x drivers installed may encounter errors when running pip install torch without specifying an index URL.

Additionally, CUDA 13.0 only supports Turing (SM 7.5) and newer GPU architectures on Linux x86_64. Maxwell and Pascal GPUs are no longer supported under CUDA 13.0. Users with these older GPUs should use the CUDA 12.6 builds instead.

CUDA 12.6 and 12.8 binaries remain available via download.pytorch.org.

Version 2.10:

# PyPI wheel used CUDA 12.x
pip install torch

Version 2.11:

# PyPI wheel now uses CUDA 13.0
pip install torch

# To get CUDA 12.8 wheels instead:
pip install torch --index-url https://download.pytorch.org/whl/cu128

# To get CUDA 12.6 wheels (includes Maxwell/Pascal/Volta support):
pip install torch --index-url https://download.pytorch.org/whl/cu126
Python Frontend
torch.hub.list(), torch.hub.load(), and torch.hub.help() now default the trust_repo parameter to "check" instead of None. The trust_repo=None option has been removed. (#174101)

Previously, passing trust_repo=None (or relying on the default) would silently download and run code from untrusted repositories with only a warning. Now, the default "check" behavior will prompt the user for explicit confirmation before running code from repositories not on the trusted list.

Users who were explicitly passing trust_repo=None must update their code. Users who were already passing trust_repo=True, trust_repo=False, or trust_repo="check" are not affected.

Version 2.10:

# Default trust_repo=None — downloads with a warning
torch.hub.load("user/repo", "model")
# Explicit None — same behavior
torch.hub.load("user/repo", "model", trust_repo=None)

Version 2.11:

# Default trust_repo="check" — prompts for confirmation if repo is not trusted
torch.hub.load("user/repo", "model")
# To skip the prompt, explicitly trust the repo
torch.hub.load("user/repo", "model", trust_repo=True)
torch.nn
Add sliding window support to varlen_attn via window_size, making optional arguments keyword-only (#172238)

The signature of torch.nn.attention.varlen_attn has changed: a * (keyword-only separator) has been inserted before the optional arguments. Previously, optional arguments like is_causal, return_aux, and scale could be passed positionally; they must now be passed as keyword arguments. A new window_size keyword argument has also been added.

# Before (2.10)
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, True, None, 1.0)

# After (2.11) — pass as keyword argument
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, window_size=(-1, 0), return_aux=None, scale=1.0)
Remove is_causal flag from varlen_attn (#172245)

The is_causal parameter has been removed from torch.nn.attention.varlen_attn. Causal attention is now expressed through the window_size parameter: use window_size=(-1, 0) for causal masking, or window_size=(W, 0) for causal attention with a sliding window of size W. The default window_size=(-1, -1) corresponds to full (non-causal) attention.

# Before (2.10)
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, is_causal=True)

# After (2.11) — use window_size instead
output = varlen_attn(query, key, value, cu_seq_q, cu_seq_k, max_q, max_k, window_size=(-1, 0))
Distributed
DebugInfoWriter now honors $XDG_CACHE_HOME for its cache directory in C++ code, consistent with the Python side. Previously it always used ~/.cache/torch. (#168232)

This avoids issues where $HOME is not set or not writable. Users who relied on ~/.cache/torch being used regardless of $XDG_CACHE_HOME may see debug info written to a different location.

Version 2.10:

# C++ DebugInfoWriter always wrote to ~/.cache/torch

Version 2.11:

# C++ DebugInfoWriter now respects $XDG_CACHE_HOME/torch (same as Python code)
# Falls back to ~/.cache/torch if $XDG_CACHE_HOME is not set
DeviceMesh now stores a process group registry (_pg_registry) directly, enabling torch.compile to trace through get_group(). (#172272)

This may break code that skips init_process_group, loads a saved DTensor (constructing a DeviceMesh with no PGs), and later creates PGs separately — during torch.compile runtime the PG lookup will fail. Users should ensure process groups are initialized before constructing the DeviceMesh.

Version 2.10:

# PGs resolved via global _resolve_process_group at runtime
mesh = DeviceMesh(...)  # PGs could be created later

Version 2.11:

# PGs now stored on DeviceMesh._pg_registry; must exist at mesh creation
dist.init_process_group(...)  # Must be called before creating mesh
mesh = DeviceMesh(...)
Distributed (DTensor)
DTensor.to_local() backward now converts Partial placements to Replicate by default when grad_placements is not provided. (#173454)

Previously, calling to_local() on a Partial DTensor would preserve the Partial placement in the backward gradient, which could produce incorrect gradients when combined with from_local(). Now, the backward pass automatically maps Partial forward placements to Replicate gradient placements, matching the behavior of from_local().

Users who relied on the previous behavior (where to_local() backward preserved Partial gradients) may see different gradient values. To ensure correctness, explicitly pass grad_placements to to_local().

Version 2.10:

# Partial placement preserved in backward — could produce incorrect gradients
local_tensor = partial_dtensor.to_local()

Version 2.11:

# Partial → Replicate in backward by default (correct behavior)
local_tensor = partial_dtensor.to_local()
# Or explicitly specify grad_placements for full control:
local_tensor = partial_dtensor.to_local(grad_placements=[Replicate()])
_PhiloxState.seed and _PhiloxState.offset now return torch.Tensor instead of int (#173876)

The DTensor RNG internal _PhiloxState class changed its seed and offset properties to return tensors instead of Python ints, and the setters now expect tensors. This makes the RNG state compatible with PT2 tracing (the previous .item() calls were not fake-tensor friendly).

Code that directly reads _PhiloxState.seed or _PhiloxState.offset and treats them as ints will break. Call .item() to get the int value. When setting, wrap the value in a tensor.

Version 2.10:

from torch.distributed.tensor._random import _PhiloxState

philox = _PhiloxState(state)
seed: int = philox.seed          # returned int
philox.offset = 42               # accepted int

Version 2.11:

from torch.distributed.tensor._random import _PhiloxState

philox = _PhiloxState(state)
seed: int = philox.seed.item()   # now returns Tensor; call .item() for int
philox.offset = torch.tensor([42], dtype=torch.int64)  # must pass Tensor
ROCm
caffe2 support is fully removed from ROCm PyTorch's hipify preprocessing. This is known as "hipify v2" behavior. (#174087, #174300, #174388, #174499, #175098)
hipify v1 background

When caffe2 and PyTorch were separate projects, the ROCm support strategies were different. For caffe2, all files and classes would be renamed following the pattern of CUDA to HIP, Cuda to Hip, cuda to hip, and so on. PyTorch did not rename classes, but would create new files following the same renaming pattern (e.g., aten/src/ATen/cuda/CUDABlas.h to aten/src/ATen/hip/HIPBlas.h). As a consequence, caffe2 had a distinct device backend named "HIP" (renamed from "CUDA") while ROCm PyTorch masquerades as the "cuda" device (torch.empty(1, device="cuda")). Once caffe2 and PyTorch projects were merged, this caused a mismatch between caffe2 expecting to use a "HIP" device while PyTorch expecting a "cuda" device. To alleviate this mismatch, "Masquerading" classes were created under aten/src/ATen/hip/impl.

  • HIPAllocatorMasqueradingAsCUDA.h
  • HIPCachingAllocatorMasqueradingAsCUDA.h
  • HIPGuardImplMasqueradingAsCUDA.h
  • HIPStreamMasqueradingAsCUDA.h These classes were often transparently utilized during ROCm PyTorch's hipify preprocessing of source files. All files under c10/ and caffe2/ were hipified using the caffe2 renaming behavior, while all other "PyTorch" files used the other strategy. The Masquerading classes would replace their CUDA counterpart during hipify preprocessing. For example, c10/cuda/CUDAStream.h's CUDAStream would be replaced by aten/src/ATen/hip/impl/HIPStreamMasqueradingAsCUDA.h's HIPStreamMasqueradingAsCUDA. These Masquerading classes call the underlying caffe2 code and create "HIP" devices, and the device would be reset to "cuda" by the Masquerading classes.
hipify v2 new behavior

Hipify v2 (#174087, #174300, #174388, #174499, #175098) makes the following changes:

  • "Masquerading" classes are deprecated. Reworked to be thin shells around existing classes, for backward compatibility.
  • Do not rename "CUDA" classes to "HIP". Only rename CUDA Runtime APIs. Files are still renamed out of place.
  • Removes caffe2 work-arounds for HIP device versus CUDA device. Great care has been taken to make this change backwards compatible. Though PyTorch today builds cleanly using hipify v2 behavior, downstream PyTorch extension projects that explicitly included Masquerading headers or called Masquerading APIs could be affected, resulting in failed builds. As an example, before backwards compatibility was realized, the xformers project had failed to build using the hipify v2 changes. A PR demonstrates the changes that were initially necessary to work around the build failures, but such changes are no longer necessary after hipify v2 BC-breaking behavior was improved.
torch.export
torch.export.export_for_training has been removed (#171714)

export_for_training was previously available as a separate API for exporting models while preserving training semantics. This function has been removed. Users should use torch.export.export instead, which returns the same graph as the previous export_for_training.

ONNX
Remove the fallback option from torch.onnx.export (#173189)

The fallback parameter has been removed from torch.onnx.export(). Previously, when fallback=True, the exporter would automatically fall back to the legacy TorchScript-based exporter if the dynamo exporter failed. This fallback was removed because it was overly complicated, required different inputs, produced different models, and hid errors from the new exporter.

Migration: Remove fallback=True (or fallback=False) from your torch.onnx.export() calls. If you need fallback behavior, implement it explicitly in your own code by catching exceptions and calling the legacy exporter separately.

# Before
torch.onnx.export(model, args, "model.onnx", dynamo=True, fallback=True)

# After
torch.onnx.export(model, args, "model.onnx", dynamo=True)
Remove overload matching logic from the ONNX dispatcher (#165083)

The custom_translation_table parameter in torch.onnx.export() no longer accepts a list of functions for each torch op. Previously, users could pass a list of overloaded ONNX functions (e.g., one for float tensors, another for bool tensors), and the dispatcher would automatically select the correct overload based on input types. This complex type-matching logic has been removed because torchlib no longer uses overloads for the same opset version.

The type of custom_translation_table changed from dict[Callable, Callable | Sequence[Callable]] to dict[Callable, Callable]. Passing a Sequence as a value now raises a TypeError.

Migration: Provide a single function per operator instead of a list of overloads. If you need type-dependent behavior, handle it inside the single function.

# Before
custom_translation_table = {
    torch.ops.aten.logical_and.default: [custom_impl_float, custom_impl_bool],
}

# After
custom_translation_table = {
    torch.ops.aten.logical_and.default: custom_impl,
}
Quantization
The PT2E quantization flow (torch.ao.quantization.pt2e and torch.ao.quantization.quantizer) has been removed from PyTorch and migrated to torchao. (#169151)

The following modules and classes have been removed:

  • torch.ao.quantization.pt2e (including DuplicateDQPass, PortNodeMetaForQDQ, export utils, graph utils, numeric debugger, lowering utilities)
  • torch.ao.quantization.quantizer (including ComposableQuantizer, EmbeddingQuantizer, X86InductorQuantizer, XPUInductorQuantizer, XNNPACKQuantizer, QuantizationSpec, QuantizationAnnotation, QuantizationConfig, etc.)

Users relying on the PT2E quantization flow should migrate to the torchao package, which now hosts these APIs.

Version 2.10:

from torch.ao.quantization.pt2e import prepare_pt2e, convert_pt2e
from torch.ao.quantization.quantizer.x86_inductor_quantizer import X86InductorQuantizer

Version 2.11:

# Install torchao: pip install torchao
from torchao.quantization.pt2e import prepare_pt2e, convert_pt2e
from torchao.quantization.pt2e.quantizer.x86_inductor_quantizer import X86InductorQuantizer

Deprecations

Linear Algebra

  • The MAGMA backend for linear algebra operations is now deprecated and will be removed in a future release. Setting torch.backends.cuda.preferred_linalg_library("magma") or retrieving a previously-set MAGMA preference will now issue a deprecation warning. cuSOLVER remains the default backend. (#172823)

    If you see any errors when using cuSOLVER that did not occur with MAGMA, please file an issue on GitHub. To silence the warning, stop explicitly selecting the MAGMA backend:

    Version 2.10:

    # No warning
torch.backends.cuda.preferred_linalg_library("magma")

    Version 2.11:

    # Issues a deprecation warning — remove this call to use the default cuSOLVER backend
torch.backends.cuda.preferred_linalg_library("magma")

  • torch.linalg.svd no longer dispatches to MAGMA. The MAGMA backend is deprecated and cuSOLVER is now used unconditionally, providing significant speedups (2x–400x depending on matrix size and batch dimensions). (#172824)

    Previously, setting torch.backends.cuda.preferred_linalg_library("magma") would route SVD through MAGMA. This setting is now ignored for SVD, and cuSOLVER is always used.

    Version 2.10:

    torch.backends.cuda.preferred_linalg_library("magma")
U, S, Vh = torch.linalg.svd(x)  # Uses MAGMA

    Version 2.11:

    # MAGMA preference is ignored; cuSOLVER is always used
U, S, Vh = torch.linalg.svd(x)  # Uses cuSOLVER

  • torch.linalg.solve_triangular and torch.triangular_solve no longer dispatch to MAGMA on CUDA. cuBLAS is now used unconditionally, providing speedups of 2x–24x for most matrix sizes (small matrices may see minor regressions of ~0.6x). (#174109)

    Version 2.10:

    torch.backends.cuda.preferred_linalg_library("magma")
torch.linalg.solve_triangular(A, B, upper=False)  # Uses MAGMA

    Version 2.11:

    # MAGMA preference is ignored; cuBLAS is always used
torch.linalg.solve_triangular(A, B, upper=False)  # Uses cuBLAS

  • torch.linalg.lstsq no longer dispatches to MAGMA. cuSOLVER/cuBLAS are now used unconditionally, providing speedups of 1.7x–620x depending on matrix size and batch dimensions. (#174779)

    Version 2.10:

    torch.backends.cuda.preferred_linalg_library("magma")
result = torch.linalg.lstsq(A, B)  # Uses MAGMA

    Version 2.11:

    # MAGMA preference is ignored; cuSOLVER/cuBLAS is always used
result = torch.linalg.lstsq(A, B)  # Uses cuSOLVER/cuBLAS
Distributed
torch.distributed.symmetric_memory.enable_symm_mem_for_group is deprecated. The store can be retrieved directly via ProcessGroup.getStore() in C++, making this call unnecessary. (#172163)

Version 2.10:

from torch.distributed.symmetric_memory import enable_symm_mem_for_group
enable_symm_mem_for_group(group)

Version 2.11:

# No longer needed — store is accessed directly from the ProcessGroup

New features

Python Frontend

  • Added native_handle property to torch.Stream, providing a unified way to retrieve the backend-specific opaque stream handle (e.g., cudaStream_t for CUDA, sycl::queue* for XPU). This is useful for passing stream handles to third-party libraries such as Triton. (#171040)

    stream = torch.accelerator.current_stream()
handle = stream.native_handle  # backend-specific stream handle
Autograd
  • Add Function.clear_saved_tensors_on_access class attribute to automatically free saved tensors after they are accessed (#173833)
torch.nn
  • Add mechanism to restore default flash attn impl after activate_flash_attention_impl (#169866)
  • Add scale for softmax to varlen attn (#171199)
Distributed
  • Add start_method option to torch.distributed.debug.start_debug_server to select the multiprocessing start method (fork, spawn, or forkserver), enabling CUDA-safe server startup (#173196)
  • Add support for periodic dumping in torch.distributed.debug (#174808)
  • Non-functional collectives (e.g. torch.distributed.all_gather) now automatically work with FakeTensorMode — meta implementations are registered at import torch time (#162119)
  • Implement NCCL 2.29 one-sided APIs for symmetric memory (#172425)
  • Bind SymmetricMemory as a torch class for use in op definitions (#174019)
  • Enable torchcomms _BackendWrapper shim layer in c10d (#174202)
  • Expose SymmetricMemory window API (#170740)
CUDA
  • Make (pinned) host memory allocations work with memory pools. (#167507)
  • Make large segment size configurable for allocation performance tuning (esp. re: Expandable Segments). (#172056)
MPS
  • Async error reporting from GPU operations (#170002, #170050) 
    import torch
x = torch.rand(10, 1, 10, device='mps')
y = x[:, [1]]
torch.mps.synchronize()  # will raise index out of bounds error
  • Added support for Metal 4 (#172229, #172230)
ROCm
  • Expose device properties clock_rate, memory_clock_rate, memory_bus_width, memory_per_block, shared_memory_per_block. (#170572)
  • Support for device-side assertions via TORCH_USE_HIP_DSA. (#172679)
  • Attention operator support on gfx1151/1152/1153 via AOTriton 0.11.2b update (#174105)
  • Enable scaled group mm on gfx950. (#173737)
  • Enable group gemm on gfx90a. (#169356)
  • Enable MIOpen backend for CTC Loss. (#170749)
  • Add hipsparseSpSV and hipsparseSpSM support for triangular solve. (#171097)
  • Support for PyTorch's StaticCudaLauncher, which provides static compilation and launching of Triton kernels. (#166492)
XPU
  • Introduce XPUGraph, a runtime optimization feature designed to reduce kernel host overhead on XPU devices, detail in: design and usage. (#166285, #174041, #174351, #174059, #174046, #166843)
torch.compile
Dynamo
  • torch.compile now supports tracing through contextlib.ExitStack and contextlib.suppress context managers, allowing code that uses these patterns to be compiled without graph breaks (#146506, #147990)
  • Added torch._dynamo.config.ignore_logging_functions config to skip arbitrary logging callables during tracing without causing graph breaks. Add functions to this set to have Dynamo treat them as no-ops during compilation (#168913)
  • Added TORCH_DYNAMO_AUTOMATIC_DYNAMIC_SHAPES=0 environment variable to globally disable automatic dynamic shapes without modifying Python code (#172334)
  • Added TORCH_COMPILE_OVERRIDE_BACKENDS environment variable for per-graph backend override, enabling binary search to find problematic compiled graphs. Supports filter syntax like ">10:eager" or "0-5:aot_eager;6-10:inductor" (#172411)
  • Added initial support for torch._dynamo.decorators.leaf_function, which allows annotating functions as leaf operations that Dynamo and AOTAutograd will not trace into (#170471)
  • Added support for tracing backward hooks on intermediate tensors, fixing cases where register_hook on non-leaf tensors would fail under torch.compile (#172126)
Inductor
  • FlexAttention supports deterministic mode, wired through both Flex and Flash backends (#173126)
  • Added range-based autotuning for custom ops, enabling selection of optimal implementations based on runtime tensor dimension values with per-range benchmarking and automatic torch.cond dispatch generation (#167617)
  • FlexAttention: Added support for low precision K/V inputs in compiled mode. Keys and Values can now be in lower precision than Queries for memory efficiency (#171761)
  • Added native ldexp lowering with libdevice.ldexp (CUDA) and std::ldexp (CPU) codegen (#171721)
  • Inductor now supports pin_memory for torch.empty (#172578)
  • Exposed triton_meta to TritonTemplate maybe_append_choice API for custom template development (#174292)
  • Added Async Pipelined Autotuning for max-autotune-gemm, which overlaps autotuning with lowering/scheduling in a subprocess to reduce compilation overhead (#170407)
  • FlexFlash: Added BlockSparse backward pass, dynamic shapes, and backward score-mod support (#170397, #170611, #171465)
  • Added FP8 (BlockWise128x128, BlockWise1x128) scaling support in Inductor Triton templates (#170748)
  • Autochunker: Added gradient accumulation support and ability to override number of chunks (#171359, #171477)
  • Added NVGEMM backend for GEMM operations using NVIDIA's native matmul library, with support for BMM, grouped GEMM, scaled MM, dynamic shapes (#171205, #171362, #172280, #172283, #172378, #172388, #172391, #172402, #172417, #172525, #172582, #172607, #174827)
torch.export
  • Add nested tensor serialization support for torch.export (#174720)
  • RNN modules (LSTM, GRU, etc.) can now be exported on GPUs (#169710)
  • Add patch to enable tracing LSTM with dynamic shapes (#168095)
ONNX
  • Added ExportableModule wrapper for ONNX export (#170810)
  • Added InputObserver to infer dynamic shapes for export (#172838)
  • Add a parameter to force the first dimension to be dynamic in InputObserver.infer_dynamic_shapes (#173533)
  • Implement while_loop (#162645)
  • Add invoke_subgraph HOP export support (#174283)
  • Expose ONNXProgram.rename_axes for renaming dims (#172032)
  • Support custom empty tensor shapes in InputObserver for multimodal LLM export (#174964)
Foreach
  • Added torch.linalg._powsum and torch._foreach_powsum as fused kernels that compute sum(abs(x)**ord) (equivalent to vector_norm without the root extraction) (#172685)

Improvements

Release Engineering
  • Upgrade to ROCm 7.2 with new Docker images, magma tarball, and binary builds (#173096, #173106, #173187, #174234)
  • Add an option to install cuda if required cuda/cudnn on windows AMI do not match (#177273)
Python Frontend
  • torch.load now produces clearer error messages when encountering miniz errors from PyTorchStreamReader, explicitly indicating that the checkpoint file is likely corrupt (#170244)
  • torch.load(map_location='meta') no longer reads storage data from the filesystem, improving performance when loading checkpoints onto the meta device (#170619)
Composability
  • Add check_out_variant and to_out_variant utilities for custom operator out variant validation. check_out_variant verifies that a custom op's out variant is compatible with Inductor's out_variant pass, and to_out_variant converts an OpOverload to its out variant. (#174473)
torch.nn
  • Add remove_duplicate parameter to nn.Module.modules() function (#174383)
  • Add support for low precision K/V inputs to nn.attention.flex_attention (#171744)
C++ Frontend
  • Added support for Float8_e8m0fnu and Float4_e2m1fn_x2 dtypes to stable ABI (#173669)
  • Added torch::stable::Tensor::layout() (#174735)
Distributed
  • Set thread name for Gloo internal loop for easier debugging (#169979)
  • Make context_parallel_shard more general (#170200)
  • Polish NCCL symmetric memory code (#170582)
  • Add MemPool support for NCCL symmetric memory backend (#171727)
  • Extend symmetric memory barrier to both LSA and GIN (#172701)
  • Implement get_offset for symmetric memory (#172044)
  • ProcessGroupNCCL: workaround for reduce_scatter with world_size=1 (#170922)
  • Add XCCL backend support for ProcessGroupWrapper (#171920)
  • Lazy import pdb only when user calls breakpoint() in torch.distributed (#171818)
  • Remove MB < PP check for GPipe pipeline schedule (#171462)
  • Pass DDP bucket cap size list for finer-grained control (#169026)
  • Enable ProcessGroup round-trip through JIT via CapsuleType (#172794)
  • Don't repeatedly log environment variables (#170399)
  • Set NCCL group desc before creating comm so it propagates (#171159)
  • ProcessGroupNCCL: use lowest rank as split color (#173687)
DTensor
  • Add OpSchema.args_meta, kwargs_meta helpers (#170358)
  • Support misc sym ops (#172268)
  • DTensor Ops: Add linearity support for neg operation (#172563)
  • Add SymInt support for DTensor mesh coordinate computation in PT2 (#169552)
  • Enable single-dim strategy for addmm and baddbmm (#172387)
  • Support uneven _StridedShard redistribution (#172266)
  • Update TP api to support single-dim strategies (#173567)
  • Initial support for decomps + sharding prop (#171652)
  • Add shard prop cache logging (#173775)
  • Optimize redistribute comms using flattened meshes (#174630)
CPU
  • Added support for FP16 half-precision GEMM via OpenBLAS on CPU, enabling faster FP16 inference (#169042)
CUDA
  • Remove _scaled_mm layout check on Blackwells (#170693)
  • Add uint16, uint32, uint64 support to JIT CUDA kernels (#174303)
  • Remove fallback paths for pinned memory allocation during CUDA graph capture (#170710)
  • Improve numerics of UpSample kernel by using accscalar_t for interpolation accumulators (#170661)
  • Reinstate error message details in CUDA_KERNEL_ASSERT_VERBOSE call in IndexKernelUtils.cu (#170913)
  • Switch order of blocked reduce in reduction_template.cuh (#173425)
cuDNN
  • Upgrade cuDNN to 9.15.1 for CUDA 13 builds (#169412)
  • Upgrade CUDA 13.0 wheels to cuDNN 9.17.1 (#173216)
  • Enhance cuDNN tensor shape checks in sdp_utils.cpp to support Blackwell GPUs (#172621)
MPS
  • Improved support for distributions operations (#172187, #172675, #173287)
  • Enabling index_fill backward pass (#174238)
  • Extended baddbmm and addbmm to integer and complex types (#170895)
  • Improved error messages for distributed ops on MPS (#173954)
  • Added MPS support for torch.special.erfcx (scaled complementary error function) (#172910)
ROCm
  • addmm behavior now takes into account preferred BLAS backend instead of forcing hipblaslt. (#174350)
  • Enable hipBLASLt on gfx1103. (#172180)
Sparse Frontend
  • torch.view_as_real and torch.view_as_complex now support sparse tensors (#164964)
  • Sparse tensor invariants check warning is now raised only once when the check is disabled, instead of on every operation (#171695)
XPU
  • Add torch.xpu._dump_snapshot API (#170186)
  • Add torch.xpu._record_memory_history API (#169559)
  • Add torch.xpu.memory_snapshot (#169442)
  • Add local_mem_size to XPU device properties (#172314)
  • Support torch.accelerator.get_device_capability on XPU (#170747)
  • Enable Triton online softmax kernels on XPU (#163251)
  • Support woq_int8 Inductor pattern on Intel GPU (#163615)
  • Add XPU ATen GEMM overloads with output dtype (#170523)
  • Support aot_inductor.emit_multi_arch_kernel on XPU (#171432)
  • Improve Inductor UT coverage for XPU (#171280, #166376, #169181, #166504)
  • Enable Triton mm template decompose_k choice for XPU (#170541)
  • Support AOTInductor standalone compile API for XPU (#171450)
Profiler
  • The memory visualizer now has a checkbox to toggle showing the trace, useful for large traces that take a long time to load (#174717). The memory profiler now exposes a new skip_actions flag to filter out specific events (#168183).
  • The profiler now exposes a post_process_timeout_s field to prevent post processing from blocking further execution (#173957).
torch.compile
Dynamo
  • Suppressed repeated "triton not found" messages during import — previously 12 identical warnings were printed (#172614)
  • fullgraph=True now recursively disables dynamo on compiled code to prevent unintentional re-invocation of torch.compile (#173080)
  • Miscellaneous smaller tracing support additions:
    • Support for Enum.__contains__ and constants (#173223)
    • Updated nn module hook handling to work with kwargs=True (#172519)
    • Support object type in dynamo tracing (#171457)
  • Add args print support to hop print (#170880)
  • Don't register einops ops with allow_in_graph (#173611)
Inductor
  • Improved heuristics for reduction kernels (#170931)
  • CUDAGraph partitioning now supports cudagraph-unsafe symints (#173159)
  • MixOrderReduction: Added low precision reduction support, non-strict mode, and avoid recompile (#169978, #171941, #174947)
  • Triton compilation timeout is now configurable and defaults to 5 minutes (lowered from previous default) (#172674)
  • User stack traces are now reported when a LoweringException occurs, making debugging easier (#171846)
  • Added B300 (Blackwell) support: GPU architecture 120a for .ptx to .fatbin compilation and cpp codegen (#174162, #172263)
  • Autotune process pool now inherits tf32 options from the parent process (#174742)
  • Epilogues can now be statically analyzed for fusion decisions (#170001)
  • Added cvt_e8m0_rceil prim with PTX lowering for SM100+ GPUs (#172497)
  • Basic comm buffer reuse for Symmetric Memory (#171909)
  • Added launch_cooperative_grid flag for cooperative reduction kernels (#167800)
  • Updated CUTLASS codegen to support torch.float8_e5m2, enabling mixed FP8 (e4m3fn x e5m2) matrix multiplication (#171167)
  • Improved mkldnn convolution layout propagation in Inductor (#169260)
  • Optimal Epilogue fusion overlapping with Async Pipelined Autotuning (#171011)
  • FlexAttention improvements: Enabled SM90 blocksparse backwards, updated configuration for Thor and DGX Spark hardware, and enabled TMA path by default on Intel GPU (#171685, #173898, #172316)
  • Added support for torchcomms lowering in inductor IR (#171634)
  • Allow int8 layout dtype for cpp gemm template on CPU (#169161)
  • Improved batch matmul codegen (#172678)
  • Improved error message in standalone_compile when there are no aot_autograd artifacts (#174086)
  • Removed unnecessary synchronize before launcher creation (#169432)
  • Removed implicit float64 upcast in Triton codegen, improving performance and reducing unnecessary precision casting (#172143)
  • Added torch.compile compatibility to FP8 SDPA using FlashAttention3, including meta registration and inductor lowering fallback for the new scaled_dot_product_flash_attention.low_p overload (#172622)
  • Replace record_function with _RecordFunctionFast in CompiledFxGraph for reduced profiling overhead (#163976)
  • Relaxed restriction on triton template mutated_inputs, allowing more flexible template usage (#170721)
  • Added combo_kernels_pointwise_only config option to exclude reduction kernels from combo kernel fusion (#174894)
  • Add a fusion region utility for grouping inductor fusible nodes for aten estimation (#170559)
  • Pallas backend: Added support for pooling with strided indexing, masked operations, random, FloorDiv, flattened indexing, welford fallback, ModularIndexing, transpose, im2col gather pattern detection, element-wise pairing, sympy min/max, FMA, automatic padding to WARPGROUP_SIZE, atomic_add store mode, TMA for OOB masking on Mosaic GPU, jax/cuda stream sync, better iter var tracking, and interleaved rope (#170014, #170145, #170221, #170222, #170232, #170595, #170616, #170627, #170738, #170741, #171449, #171475, #171518, #171539, #171567, #172306, #173840, #174249, #174797)
  • Add per-graph inductor config override for debugging/bisecting (#174228)
torch.fx
  • torch.fx.symbolic_trace now supports tracing HigherOrderOperators that do not take callable arguments (#173839)
  • Rename hint_int to size_hint, support size_hint in user code. (#171944)
  • Add metadata hook for all nodes created in runtime_assert pass (#173970)
  • Add _disable_torch_fn_metadata_mode option to make_fx and aot_export_joint_with_descriptors (#172087)
  • Add nested value-type opaque object support (#169845)
torch.export
  • from_node provenance information is now preserved when serializing exported programs (#171726)
  • Bitwise shift operations are now supported in the export serializer (#167913)
  • Improve leak detection in non-strict export mode (#172597)
Quantization
  • Use expm1 for computing quantized ELU, improving numerical stability (#173968)
ONNX
  • Implement torch.sym_sum and torch.sym_ite (#170263)
  • Raise an error if there are duplicated input/output names (#173077)
  • Refactor optimize and version conversion logic (#173185)
Optimizer
  • Optimizer graph capture check now supports XPU devices in addition to CUDA (#172759)
DevX
  • The spin lint command now supports pass-through arguments to lintrunner, including --take, --skip, and --tee-json flags, giving developers more control over which linters run (#169373)
Ahead-Of-Time Inductor (AOTI)
  • Better error message for mixed device tensors (#173982)
  • Support mixed-device constants (#169504)
  • Change cpp_kernel_name to public API to match AOTI shim gen; add mm_type_out to AOTI fallback kernel (#174489)

Bug fixes

Release Engineering
  • Fixed macOS wheel metadata where setuptools misinterpreted the platform version string, producing incorrect wheel tags for macOS arm64 builds (#173541)
  • Fixed incorrect wheel naming (#173945)
  • Fixed macOS arm64 libtorch release upload failure (#175100)
  • Fix pep517 release handling (#175635)
Python Frontend
  • Fixed a bug where torch.load with FakeTensorMode or skip_data context would compute incorrect storage sizes (#170618)
  • Fixed PrivateUse1 backend aliasing during deserialization so custom backends are correctly recognized when loading checkpoints (#165456)
  • Fixed torch.ops.aten.index.Tensor to properly raise an IndexError when called with an empty indices list, instead of producing undefined behavior (#174009)
Autograd
  • Fixes absolute tolerance scaling for complex backpropagation in torch.autograd.gradcheck when fast_mode=True (#166386)
Complex Frontend
  • Fixed torch.view_as_complex() not working on the memory layout produced by .contiguous() after .transpose() (#169780)
Composability
  • Fix torch.bucketize crash during torch.export when test_elements is a scalar (#170751)
  • Fix MaxUnpool crash when input tensors are small (#169359)
Dataloader
  • Fix DataLoader to respect overridden __getitem__ in Subset subclasses (#163961)
Nested Tensor (NJT)
  • Fix NestedTensor min/max operations for integer dtypes (#167685)
torch.nn
  • Fix Illegal Memory Access in FlashAttention 2 by syncing with upstream (#174114)
  • Propagate max_q and max_k for varlen_attn (#173681)
C++ Frontend
  • Fixed bug in stable ABI shim to handle special types (SymInt, MemoryFormat, ScalarType, Layout) correctly (#174734)
Distributed
  • Add half precision binding for MPI backend (#170074)
  • Fix incorrect boolean logic in std::string::find method in c10d (#170057)
  • Fix _set_pg_timeout not working for Gloo backend (#167052)
  • Fix DeviceMesh corner case for coalesce in cute layout and mesh slicing (#169454)
  • Fix Context Parallel flex_input_fn argument unwrapping issue (#170201)
  • Fix FSDP _unshard() passing Stream instead of Event (#170525)
  • Ensure threadblock size >= world size in CUDA symmetric memory barrier (#170785)
  • Fix ProcessGroupGloo CUDA tensor stream handling with futures (#170812)
  • Fix env variable to retrieve HCA list for NVSHMEM (#170891)
  • Fix FSDP split_with_sizes_copy() missing dim argument (#169173)
  • Fix cross-thread work registry lookup in wait_tensor (#171614)
  • Fix fully_shard arg typehint inconsistency (#171574)
  • Fix Flight Recorder default buffer size inconsistency (#172843)
  • Remove mixed dtype rejection for clip_grad_norm to align with documentation (FSDP) (#173641)
  • Fix all-reduce strides in compiled code (#171616)
  • Fix ProcessGroupWrapper missing method forwarding (#173599)
  • Fix syntax for suppression comments. (#167088)
Distributed Checkpoint (DCP)
  • Fix TypedStorage deprecation warning in distributed checkpoint (#170759)
  • Fix typo in variable name from 'statetful_sd' to 'stateful_sd' (#171292)
DTensor
  • Preserve Partial(max/min) reduce op type on torch.max/torch.min output DTensors (#170203)
  • Prevent pointwise operations between Partial DTensors with different reduce ops (#170209)
  • Fix OpInfo.schema type and add asserts (#170790)
  • Fix _StridedShard(sf=) bug in single dim strategy (#171942)
  • Fix incorrect Tensor Meta Population (#172304)
  • Single_dim fix symint + _create_expanded_strategy (#172421)
  • Single dim fix inplace op expansion (#172477)
  • Fix single-dim output_meta validation (#172293)
  • Fix redistribute cost crashing on non-participating ranks (#172478)
  • Fix t() sharding strategy for 1D tensors (#173964)
  • Fix unsupported op error (#170889)
  • Fix DTensor honor single-dim RuntimeSchemaInfo (#174312)
  • Fix device_mesh extraction from kwargs (#173489)
  • Fix StridedShard usage conflict with shard order (#174831)
  • Fix bucketize with Partial inputs (#173937)
  • Fix embedding_dense_backward cache key missing num_weights (#174727)
FullyShardedDataParallel2 (FSDP2)
  • Fix mixed DTensor error with nested FSDP and activation checkpoint (#171779)
CUDA
  • Don't false-positive on record_stream and account for 0-element tensors in CUDA stream sanitizer (#172562)
  • Fix failures due to launch bounds for ctc_loss_gpu_template on SM12+ (#172447)
  • Fix the torch.Stream context manager reentrance (#176568)
cuDNN
  • Disable a cuDNN Convolution engine preemptively (#171747)
MPS
  • Fixed non-contiguous grid sampler on MPS (#171619)
  • Fixed large reductions when compiling for MPS (#171479)
  • Fixed MPS Inductor tanh implementation (#172406)
  • Fixed complex to real power scalar on MPS (#174147)
  • Fixed masked op logic in MPS Inductor (#170134)
  • Fixed orgqr race condition on MPS (#174143)
  • Fixed 2-pass SDPA memory corruption by forcing float accumulators, resolving nondeterministic/corrupt results with bf16/fp16 and GQA when seq_len > 1023 (#174945)
  • Fix half-precision type mismatches in Metal shader codegen (#176436)
ROCm
  • Sliding window attention NaN issue is fixed by AOTriton 0.11.2b. (#173204, #174105)
  • Increase the event_name attribute of autograd's profiler_util.py to avoid truncation of long HIP events. (#174366)
  • Cholesky operator via MAGMA was missing a sync operation. (#172112)
  • Updated patched libdrm in bundled release wheels to avoid missing amdgpu.ids warning and properly return AMDGPU marketing names. (#174811)
  • Fix fake_quantize undefined behavior with inf. (#171777)
  • Fix deterministic scan kernel edge case. (#170763)
  • Use torch's caching allocator for CK workspaces, for better memory behavior and hipGraph capture. (#172311)
  • Fixes grouped gemm 2d2d for edge cases with uninitialized data. (#174314)
Sparse Frontend
  • Fixed torch.sparse.spdiags crashing with zero-dimension shapes (#174052)
torch.func
  • Fixed GradTrackingTensor.tolist() not working on MPS (macOS) device tensors when used inside torch.func.grad or other function transforms (#171317)
XPU
  • Fix T5 model SDPA pattern matcher on XPU (#171774)
  • Fix torch.xpu.memory_allocated / torch.xpu.memory_reserved reporting incorrect memory sizes (#171453)
torch.compile
Dynamo
  • Fixed memory leaks: cleared weakrefs from memos/guards after compilation (#165367), cleared weak references from FakeTensorMode after compile (#171209), fixed CUDA memory usage for CPU-only compile (#163841)
  • Fixed overguarding on OrderedSet, set, and frozenset with activation checkpointing (#169535, #170291)
  • Fixed MATCH_MAPPING, MATCH_KEYS, and MATCH_SEQUENCE opcodes for Python pattern matching (match/case) support (#173085, #173086, #173087)
  • Handle List/Dict comprehension graph breaks for Python 3.12+, including nested comprehensions (#173558, #174413)
  • Fixed support for self-referential lists and dicts (#173672, #174498)
  • Fixed share_memory_ compile failure (#171162)
  • Fixed defaultdict default factory and union functionality (#168028)
  • Fixed property setter on MutableMapping subclasses (#173184)
  • Various tensor subclass fixes: subclass handling (#170871), sequence conversion (#172103), metadata propagation for in-place ops (#167583)
  • Correctly pass is_inference in the cudagraphs torch.compile backend (#174713)
AOTDispatcher
  • Fixed effect token handling when a graph contains subgraphs without tokens (#173226)
Inductor
  • Avoid TMA assert and honor descriptor min block sizes in mix-order (#170949)
  • Prevent fusing atomic scatter ops with dependent reads (#172301)
  • Combo kernel fixes: missing store masks, persistent_reduction heuristics, FusedMixOrderReductions grouping, and scalar store broadcast shape mismatch (#168939, #169509, #169721, #172658)
  • Fix torch.cond stride mismatch when subgraph outputs are padded (#169963)
  • Fix view_copy decomposition to handle dtype size changes (#171442)
  • Fix output shape mismatch for aten.isin with scalar inputs (#171272)
  • Fix segfault with DeviceContext and python scalar (#172660)
  • Fix WeakDeps for clone ops in scheduler (#173703)
  • Fix strides for aten.uniform (#170794)
  • Handle negative scaling factors in upsample_nearest (#171151)
  • FlexAttention fixes: non-power-of-2 head dimensions in decode, score mod captured grad dtype, GQA/MQA, force last dim contiguous, deterministic semantics, and TMA availability fallback (#164931, #170842, #171271, #172690, #174251, #174427)
  • Fix compatibility with new TF32 API by removing legacy allow_tf32 usage from inductor internals (#173731)
  • Fix out of shared memory issue with too high num_stages (#170071)
  • Fix cooperative reduction deadlocks due to too many CTAs (#170162)
  • Fix select_scatter dtype assertion error (#171311)
  • Fix bug in aten.pow lowering (#170960)
  • Make silu output match eager mode in inductor (#171723)
  • Fix constant folding crash on 0-d complex tensors (#172181)
  • Fix bucketize crash in compile mode (#171595)
  • Fix OverflowError when truncating infinity numbers (#166636)
  • Fix non-contiguous valid reshape handling for dynamic shapes (#173062)
  • Fix strides for narrow_copy decomposition (#173043)
  • Fix race condition in FxGraphCache when reading temp files (#172144)
  • Fix dynamic shapes infinity check (#173426)
  • Fix DDE in inductor/scheduler fusion (#173412)
  • Fix clone removal for conjugated tensors in remove_noop_ops (#172160)
  • Fix edge case with next_power_of_2 (#174330)
  • Fix conditions to apply TMA (#174480)
  • Fix CUTLASS import error (#174924)
  • Fix persistent reduction codegen (#174928)
  • Fix lowering fallback for _pdist_forward and _pdist_backward (#170959)
  • Fix coalesce analysis to work with symints (#161819)
  • Fix libdevice.pow type mismatch in Triton codegen (#173685)
  • Fix missed symbol propagation in cudagraphs partition (#174729)
  • Fix dtype issue in exclusive_scan_decoupled_lookback_64 (#171153)
  • Skip fuse attention on fp32 if not tf32 (#172951)
  • Fix RuntimeError in FxGraphCachePickler.dumps() for unpickleable pybind11 objects (#173577)
  • Fix loop split optimization when nodes have mismatched var_ranges (#173607)
  • Fix error parameter handling (#170952)
  • Guard n_spills None before threshold comparison in Triton heuristics (#169940)
  • Autotuning fixes: TMA workspace pickling in async-pipelined mode, synthetic offsets for grouped MM, and GEMM inputs that are views of the same buffer (#173089, #171316, #171363)
  • Fix reduction hint for inner-dimension reductions (#167472)
  • Fix crash when Triton compile fails in identify_mutated_tensors() (#170808)
  • Handle unbacked symbols in should_use_persistent_reduction (#172534)
  • Pallas backend: Multiple bug fixes including store for scalar values, index_put, scatter, cat, mean reduction codegen, negative index handling, iteration variable ordering, batch norm broadcasting, scalar store shape mismatch, square buffer transpose detection, strided tensor handling, and rope (#170026, #170027, #170028, #170464, #170593, #170594, #170653, #171215, #171476, #171504, #171571, #171581, #171612, #174791)
  • Fix use_compute_types parameter name mismatch in CPP backend to_dtype() that caused TypeError when enabling emulate_precision_casts=True on CPU (#168125)
  • Fix Invalid Memory Access in TMA mm kernel by limiting GEMM to valid MNK dimensions (#171871)
  • Guard against zero timing in combo kernel benchmarking to prevent division by zero (#172918)
  • XPU: Fix SDPA pattern matcher for T5 models (#171774)
  • Fix extern kernel kwargs with IR nodes not being materialized in FXIR backend, causing errors with ops like torch.segment_reduce (#172981)
  • Fix torch.isin compile shape for scalar test_elements (#172531)
  • Fix kernel benchmark runner (#169976)
torch.fx
  • Fix torch.fx.symbolic_trace to_folder with torch.nn.Sequential modules (#169279)
  • Fix Node.type pickling in torch.fx (#169172)
  • Raise ValueError for invalid fusion strategy and add test (#171573)
  • Fix typos (#171042)
  • Fix input mutation handling for subclasses (DTensor copy_) (#170467)
torch.export
  • Fix graph signature mutation when unlifting exported programs (#170461)
  • Fix tensor name inconsistency when round-tripping through torch.export.save and torch.export.load (#171954)
  • Fix handling of incomplete tensors (cuDNN packed format) in torch.export (#172805)
Quantization
  • Fix incorrect dilation check in quantized MaxPool3d that could produce negative values (#171790)
  • Fix incorrect values displayed in quantization error messages and log strings (#171868)
  • Add validation for out_channels in quantized ConvTranspose modules (#171628)
  • Fix crash when creating quantized tensors with empty dimensions (#163487)
  • Fix context cache for FP8 quantized linear operations where weight scales could become invalid (#172553)
ONNX
  • Handle complex initializers (#170231)
  • Fix export of torch.cdist with dynamic axes (#172758)
  • Fix InputObserver.infer_arguments with empty caches (#174205)
Foreach
  • Fixed _foreach_copy_ producing incorrect results when destination tensors have mixed dtypes (#173531)
  • Fixed _foreach_max returning incorrect results when tensors contain negative values, by initializing output with lowest() instead of zeros (#173241)
  • _foreach_norm now properly raises an error when computing infinity norm on empty tensors, instead of returning undefined results (#173238)
  • Fixed _foreach_norm to support symbolic tensors with dynamic shapes (#174026)
Ahead-Of-Time Inductor (AOTI)
  • Fix import error in aoti load (#173751)
  • Fix mixed-device zero-size constant indexing error (#172748)

Performance

Python Frontend
  • Added __slots__ to pytree TreeSpec dataclasses, reducing memory usage and improving attribute access speed (#172172)
Composability
  • Multiple optimizations to symbolic shape reasoning, including faster symbol sorting, reduced redundant hint computations, and optimized construction of relational expressions (#174615, #174655, #174664, #174652, #174665, #174662)
Distributed
  • Sort mempool registrations via allocation-time counter for CUDA mempools (#167662)
  • Improve _get_param_to_fqns from O(N^2) to O(N) in FSDP (#174675)
CUDA
  • Speedup grouped_mm a bit (#170802)
  • Add fast memory snapshot option which skips traces (#173949)
  • Allow all 10.x compute capabilities to use vec8 kernel for higher realized memory bandwidth (#174362)
MPS
  • Migrated atan2 to native MPS Metal kernel (#173405)
  • Migrated pow_tensor_scalar and reciprocal to Metal shaders (#170077)
  • Reimplemented Cauchy distribution with native Metal kernel (#174062)
  • Rewrote log_normal and geometric distributions as Metal shaders (#174189)
  • Migrated grid_sampler_2d to Metal (#174343)
ROCm
  • Revert MIOpen channels last support back to opt-in using the environment variables PYTORCH_MIOPEN_SUGGEST_NHWC=1 and PYTORCH_MIOPEN_SUGGEST_NHWC_BATCHNORM=1. (#170780)
  • New fx pass to reduce atomic contention. (#168073)
  • TopK performance improvements; single-block warp-level compaction (#171940), warp merge sort (#170029).
  • Enable fastSpecializedAtomicAdd for gfx950, improving performance of index-based operators like embedding bag, sampling, and scatter/gather. (#170330)
  • Optimize radix select by caching data on shared memory. (#172517)
  • Optimize reduction operator launch configuration for better performance. (#173576)
  • Improvements to inductor reduction kernel heuristics for MI350. (#170931)
XPU
  • Optimize int_mm performance on Intel GPU when mat2 is non-contiguous (#169555)
  • Enable static Triton kernel launcher for XPU backend to reduce model compilation time (#169938)
torch.compile
Dynamo
  • Various compile time improvements: caching for inspect.signature, var_getattr, attr source construction, and higher-order ops; fast paths for bind_args, GET_ITER on tuples, and tree_map on namedtuples; lazy variable tracker optimizations (#170100, #169959, #173582, #174437, #174438, #174141, #174020, #174130, #174901, #174598)
Inductor
  • FlexAttention performance: Improved flex decoding by avoiding unbalanced splitting, support multi BLOCK_M for flex-decoding, elide score-mod when not needed, and avoid materializing lse grad (#167935, #170343, #172380, #173481)
  • Compile-time improvements: optimized reorder/sink passes, removed expensive dynamo_timed calls, used _has_symbolic_sizes_strides more pervasively, overlapped template fusion with async_compile, and optimized Triton template heuristics (#169370, #169661, #169662, #170444, #170565, #172249, #174408)
  • Apply gumbel max trick for sampling (#171143)
  • Improved combo kernels design, add per-sub-kernel block dimensions for combo kernels and flattened grid dispatch (#171671, #172527)
  • Rewrote avg_pool2d as a reduction for improved performance (#167228)
  • Optimized tensor broadcasting (#170772)
  • Reduced autotune search space for pointwise Triton kernels (#169508)
  • Added BLOCK_K=64 and 128 autotuning configs for Triton conv, improving performance for larger channel sizes (#174752)
torch.fx
  • Improve node index lookup performance in FX graphs by using an index lookup map (#173385)
Quantization
  • Use oneDNN primitive for FP8 quantized convolution on x86, replacing the previous reference kernel (#172551)
Optimizer
  • Use fused multiply-add (FMA) instructions for fused Adam and AdamW on CUDA, improving numerical accuracy and performance (#173224)
  • Improve performance of fused Adam, AdamW and SGD by not using double compute (#173227)
Foreach
  • Enabled fast path for _foreach_addcmul and _foreach_addcdiv when tensor1 is a 0D (scalar) tensor (#172731)

Documentation

Release Engineering
  • Fixed stable C++ docs rendering (#171957)
  • Updated pytorch_sphinx_theme2 version to 0.4.3 (#174806)
  • Updated pytorch_sphinx_theme2 version to 0.4.6 (#177562)
Python Frontend
  • Clarified torch.unique behavior when using the dim parameter (#171608)
  • Clarified torch.as_tensor signature to document that dtype and device are keyword-only arguments (#173073)
  • Fixed torch.tensordot documentation (#173893)
  • Added dedicated docstring for torch.Tensor.permute to clarify it accepts variadic arguments unlike torch.permute (#170689)
Autograd
  • Improve torch.autograd.set_multithreading_enabled docs (#170204)
Nested Tensor (NJT)
  • Add warning about inactive development to NJT docs (#172645)
C++ Frontend
  • Added C++ docs for torch::stable namespace (#170912)
Sparse Frontend
  • Fixed incorrect gradient support documentation for torch.sparse.mm and torch.sparse.addmm (#174039)
XPU
  • Update XPU Get Started guide with new client GPU and formatting (#169810)
  • Document previous version of Torch XPU installation (#174453)
  • Update previous version 2.10 installation in XPU Get Started guide (#176141)
torch.fx
  • Add documentation for previously undocumented functions (#170581)
  • Remove outdated jit files (#173015)
Quantization
  • Update tensor_attributes.rst with additional torch.float4_e2m1fn_x2 dtype documentation (#170448)
ONNX
  • Change warning to debug log for missing annotations (#172247)
Optimizer
  • Improved EMA (Exponential Moving Average) equation documentation in optimizer docs (#172423)

Security

Python Frontend
  • Fixed a ZipSlip directory traversal vulnerability in torch.hub that could allow malicious zip files to extract files outside the target directory. torch.hub now validates all extracted paths and raises a ValueError if an archive attempts to write outside the expected folder. (#171754)

Developers

Release Engineering
  • Clarified needs-repro guidelines and edge cases for issue triage (#174028)
  • Updated tlparse instructions in PT2 bug report template (#172392)
  • Remove +ptx from CUDA 13.0 builds (#175567)
  • Update inductor CI jobs to CUDA 13.0 (#175826)
  • Upgrade ROCm CI to 7.2 (#173188)
  • Switch vLLM test and benchmark workflows to CUDA 13.0 (#175393)
  • Windows override AMI pre-installed cudnn (#177027)
  • Unpin cuda-bindings dependencies (#176042)
  • Stop using G3 runners (#175938)
Autograd
  • Fix grad_outputs type hints and parameter descriptions for a few autograd APIs (#164838)
Distributed
  • Fix USE_NCCL=0 build failure in nccl_dev_cap.hpp (#171694)
DTensor
  • Add DTensor performance benchmarks for collectives, from_local/to_local, and backward passes (#171576)
  • Add DTensor benchmarks for miscellaneous dispatch paths (#171847)
CPU
  • Fixed compilation failure with GCC 14.2.0 on ARM SVE targets (e.g., -march=armv8-a+sve+bf16 on Debian 13) by broadening the GCC version workaround for SVE compilation (#174647)
CUDA
  • Cleanup at::numeric_limits (#171111)
  • Move EventPool::Event to c10 (#158220)
  • Reuse CUDAEventPool in CUDA caching host allocator (#168345)
  • Move CUDAEvent to c10 (#158219)
cuDNN
  • Upgrade cuDNN to 9.19 for 12.8 and 13.0 wheels (#174310, #175547)
MPS
  • Migrated _local_scalar_dense_mps to DispatchV2 (#172967)
ROCm
  • Fix unused-result warning in UniqueCub.cu (#174203)
  • Use rocm_sdk preloaded libraries for hiprtc and amdhip64. (#169855)
  • Fix torch.utils.cpp_extension build folder accelerator detection ROCm (#170784)
  • Unify hipBLASLt architecture lists into common hook methods. (#172791)
XPU
  • Switch Intel Triton compiled kernel format from spv to zebin (#167972)
torch.compile
Inductor
  • Improved cudagraph logging: updated partition format, added tree logging, and [compile_id] prefix style (#170217, #170218, #174110)
  • Added Proton profiling support for inductor (#171192)
  • Added support for dumping input tensors for Triton kernels generated with Inductor backend (#171575)
  • Inductor compiler bisector: added run mode for debugging and cudagraph application (#170717, #172461)
  • Autotune configurability: number of choices displayed is now settable via environment variable, and rep options are easily overridable (#171429, #171629)
  • Added template codegen/emission override hooks for external template buffers (Helion integration) (#174148)
  • fx_graph_runnable improvements: fixed single quotes in envvars, missing symints, nested triton kernels, and global constexpr support (#173704, #174035, #174038, #174533)
  • Added perfetto trace events to graph pass application with envvar to disable (#172460)
torch.fx
  • GraphModule.print_readable() improvements: new additional_meta argument for displaying additional node metadata (#173734), long annotations are now truncated for readability (#173119), and fix trailing whitespace with inner graphs (#172644)
  • GraphPickler improvements: support for custom ignored metadata field keys (#172587), a debug_dumps method for debugging pickle failures (#173675), respecting __getstate__ for GraphModule serialization (#173810), and automatic fallback to dill if available (#173801)
  • Stack traces on invoke_subgraph nodes now point to the original model code for easier debugging (#170927)
  • Add process group support to fx_graph_runnable (#173932)
  • Strict type coverage in dispatch and partial subclass (#171808)
torch.export
  • Add strobelight profiling support for the export process (#174606)
Mobile
  • Fix spin lint on arm (#172965)
View originalPermalink
v2.10.0

PyTorch 2.10.0 Release

PyTorch 2.10.0 Release Notes

Highlights

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Dataloader Frontend
  • Removed unused data_source argument from Sampler (#163134). This is a no-op, unless you have a custom sampler that uses this argument. Please update your custom sampler accordingly.
  • Removed deprecated imports for torch.utils.data.datapipes.iter.grouping (#163438). from torch.utils.data.datapipes.iter.grouping import SHARDING_PRIORITIES, ShardingFilterIterDataPipe is no longer supported. Please import from torch.utils.data.datapipes.iter.sharding instead.
torch.nn
  • Remove Nested Jagged Tensor support from nn.attention.flex_attention (#161734)
ONNX
  • fallback=False is now the default in torch.onnx.export (#162726)
  • The exporter now uses the dynamo=True option without fallback. This is the recommended way to use the ONNX exporter. To preserve 2.9 behavior, manually set fallback=True in the torch.onnx.export call.
Release Engineering
  • Rename pytorch-triton package to triton (#169888)

Deprecations

Distributed
  • DeviceMesh
    • Added a warning for slicing flattened dim from root mesh and types for _get_slice_mesh_layout (#164993)

We decided to deprecate an existing behavior which goes against the PyTorch design principle (explicit over implicit) for device mesh slicing of flattened dim.

Version <2.9
import torch
from torch.distributed.device_mesh import

device_type = (
    acc.type
    if (acc := torch.accelerator.current_accelerator(check_available=True))
    else "cpu"
)
mesh_shape = (2, 2, 2)
mesh_3d = init_device_mesh(
    device_type, mesh_shape, mesh_dim_names=("dp", "cp", "tp")
)

mesh_3d["dp", "cp"]._flatten()
mesh_3["dp_cp"]  # This comes with no warning
Version >=2.10
import torch
from torch.distributed.device_mesh import

device_type = (
    acc.type
    if (acc := torch.accelerator.current_accelerator(check_available=True))
    else "cpu"
)
mesh_shape = (2, 2, 2)
mesh_3d = init_device_mesh(
    device_type, mesh_shape, mesh_dim_names=("dp", "cp", "tp")
)

mesh_3d["dp", "cp"]._flatten()
mesh_3["dp_cp"]  # This will come with a warning because it implicitly change the state of the original mesh. We will eventually remove this behavior in future release. User should do the bookkeeping of flattened mesh explicitly.
Ahead-Of-Time Inductor (AOTI)
  • Move from/to to torch::stable::detail (#164956)
JIT
  • torch.jit is not guaranteed to work in Python 3.14. Deprecation warnings have been added to user-facing torch.jit API (#167669).

torch.jit should be replaced with torch.compile or torch.export.

ONNX
  • The dynamic_axes option in torch.onnx.export is deprecated (#165769)

Users should supply the dynamic_shapes argument instead. See https://docs.pytorch.org/docs/stable/export.html#expressing-dynamism for more documentation.

Profiler
  • Deprecate export_memory_timeline method (#168036)

The export_memory_timeline method in torch.profiler is being deprecated in favor of the newer memory snapshot API (torch.cuda.memory._record_memory_history and torch.cuda.memory._export_memory_snapshot). This change adds the deprecated decorator from typing_extensions and updates the docstring to guide users to the recommended alternative.

New Features

Autograd
  • Allow setting grad_dtype on leaf tensors (#164751)
  • Add Default Autograd Fallback for PrivateUse1 in PyTorch (#165315)
  • Add API to annotate disjoint backward for use with torch.utils.checkpoint.checkpoint (#166536)
Complex Frontend
  • Add ComplexTensor subclass (#167621)
Composability
  • Support autograd in torch.cond (#165908)
cuDNN
  • BFloat16 support added to cuDNN RNN (#164411)
  • [cuDNN][submodule] Upgrade to cuDNN frontend 1.16.1 (#170591)
Distributed

  • LocalTensor:

    • LocalTensor is a powerful debugging and simulation tool in PyTorch's distributed tensor ecosystem. It allows you to simulate distributed tensor computations across multiple SPMD (Single Program, Multiple Data) ranks on a single process. This is incredibly valuable for: 1) debugging distributed code without spinning up multiple processes; 2) understanding DTensor behavior by inspecting per-rank tensor states; 3) testing DTensor operations with uneven sharding across ranks; 4) rapid prototyping of distributed algorithms. Note that LocalTensor is designed for debugging purposes only. It has significant overhead and is not suitable for production distributed training.
    • LocalTensor is a torch.Tensor subclass that internally holds a mapping from rank IDs to local tensor shards. When you perform a PyTorch operation on a LocalTensor, the operation is applied independently to each local shard, mimicking distributed computation (LocalTensor simulates collective operations locally without actual network communication.). LocalTensorMode is the context manager that enables LocalTensor dispatch. It intercepts PyTorch operations and routes them appropriately. The @maybe_run_for_local_tensor decorator is essential for handling rank-specific logic when implementing distributed code.
    • To get started with LocalTensor, users import from torch.distributed._local_tensor, initialize a fake process group, and wrap their distributed code in a LocalTensorMode context. Within this context, DTensor operations automatically produce LocalTensors.
    • PRs: (#164537, #166595, #168110,#168314,#169088,#169734)

  • c10d:

    • New shrink_group implementation to expose ncclCommShrink API (#164518)
Dynamo
  • torch.compile now fully works in Python 3.14 (#167384)
  • Add option to error or disable applying side effects (#167239)
  • Config flag (skip_fwd_side_effects_in_bwd_under_checkpoint) to allow eager and compile activation-checkpointing divergence for side-effects (#165775)
  • torch._higher_order_ops.print for enabling printing without graph breaks or reordering (#167571)
FX

  • Added node metadata annotation API

  • Disable preservation of node metadata when enable=False (#164772)

  • Annotation should be mapped across submod (#165202)

  • Annotate bw nodes before eliminate dead code (#165782)

  • Add logging for debugging annotation (#165797)

  • Override metadata on regenerated node in functional mode (#166200)

  • Skip copying custom meta for gradient accumulation nodes; tag with is_gradient_acc=True (#167572)

  • Add metadata hook for all nodes created in runtime_assert pass (#169497)

  • Update gm.print_readable to include Annotation (#165397)

  • Add annotation to assertion nodes in export (#167171)

  • Add debug mode to print meta in fx graphs (#165874)

Inductor
  • Add experimental Pallas TorchInductor backend. (#166822)
  • Add Pallas TPU backend support. (#167774)
  • Add Flash Attention support to FlexAttention. (#161118)
  • Add deterministic mode for Inductor compilation. (#163589) (#165950) (#164532)
  • Enable custom op autotune decompositions and parameter tuning. (#164212) (#167193)
  • Expose torch.compiler.config.force_disable_caches as a public API. (#166699)
  • Add HOP for additional control dependencies to enforce explicit scheduling. (#164568)
  • Add Inductor Lite Mode (#167115)
  • Add distributed autotuning support (#163369)
  • Add Native matmul support to inductor (#157743)
Ahead-Of-Time Inductor (AOTI)
  • Integrate AOTI as a backend. (#167338)
  • Add AOTI mingw cross compilation for Windows. (#163188)
MPS
torch.nn
ONNX
  • A new testing module torch.onnx.testing with a testing utility assert_onnx_program (#162495)
Profiler
  • Add scope for RecordFunctionFast (#162661)
Quantization

  • Add _scaled_mm_v2 API (#164141)

  • Add scaled_grouped_mm_v2 and python API (#165154)

  • Add embedding_bag_byte_prepack_with_rowwise_min_max and embedding_bag_{2/4}bit_prepack_with_rowwise_min_max (#162924)

  • Add MXFP4 support for _scaled_grouped_mm_v2 via. FBGEMM kernels (#166530)

Release Engineering
ROCm
  • Enable grouped GEMM via regular GEMM fallback (#162419)
  • Enable grouped GEMM via CK (#166334, #167403)
  • Enable ATen GEMM overload for FP32 output from FP16/BF16 inputs (#162600)
  • Support torch.cuda._compile_kernel (#162510)
  • Enhanced Windows support
  • load_inline (#162577)
  • Enable AOTriton runtime compile (#165538)
  • AOTriton scaled_dot_product_attention (#162330)
  • Add gfx1150 gfx1151 to hipblaslt-supported GEMM lists (#164744)
  • Add scaled_mm v2 support. (#165528)
  • Add torch.version.rocm, distinct from torch.version.hip (#168097)
XPU
  • Support ATen operators scaled_mm and scaled_mm_v2 for Intel GPU (#166056)
  • Support ATen operator _weight_int8pack_mm for Intel GPU (#160938)
  • Extend SYCL support in PyTorch CPP Extension API to allow users to implement new custom operators on Windows (#162579)
  • Add API torch.xpu.get_per_process_memory_fraction for Intel GPU (#165511)
  • Add API torch.xpu.set_per_process_memory_fraction for Intel GPU (#165510)
  • Add API torch.xpu.is_tf32_supported for Intel GPU (#163141)
  • Add API torch.xpu.can_device_access_peer for Intel GPU (#162705)
  • Add API torch.accelerator.get_memory_info for Intel GPU (#162564)

Improvements

Build Frontend
  • Abort explicitly requested CUDA build if toolkit could not be found (#166982)
  • RISC-V build improvements (#166602, #167071, #165717)
  • Allow building with arbitrary BLAS library (#166333)
  • Allow building with LeakSanitizer (#158686)
Composability
  • If you are using the torch.compile(backend="aot_eager") backend, it should now give bitwise equivalent results in eager. Previously it sometimes would not due to extra compile-only decompositions running (#165910)
  • Some dynamic shape errors were changed to recommend using torch._check over torch._check_is_size (#164889,
  • Some unbacked (dynamic shape) improvements (#162652, #169612)
  • Some bugfixes for symbolic float handling in compile (#166573, #162788)
C++ Frontend
  • Changed TORCH_CHECK_{COND} behavior to be non-fatal (#167004)
  • Migrated TypeTraits, TypeList, Metaprogramming, DeviceType, MemoryFormat, Layout, version.h, and CppTypeToScalarType to torch::headeronly (#167386, #163999, #168034, #165153, #164381, #167610)
  • Bumped libfmt submodule version to 12.0.0 (#163441)
CUDA
  • Make torch.cuda.rng_set_state and torch.cuda.rng_get_state work in CUDA graph capture. (#162505)
  • Enable templated kernels (#162875)
  • Enable pre-compiled kernels (#162972)
  • Add CUDA headers automatically (#162634)
  • Remove outdated header_code argument (#163165)
  • Prevent copies of std::vector in CUDA ForeachOps (#163416)
  • Implement cuda-python CUDA stream protocol (#163614)
  • Remove outdated checks and docs for cuBLAS determinism (#161749)
  • Cleanup old workaround code in launch_logcumsumexp_cuda_kernel (#164567)
  • Add a compile-time flag to trigger verbose logging for device-side asserts (#166171)
  • Support SM 10.3 in custom CUTLASS matmuls (#162956)
  • Enable CUTLASS matmuls on Thor (#164836)
  • Add per_process_memory_fraction option to PYTORCH_CUDA_ALLOC_CONF (#161035)
  • Support nested memory pools (#168382)
  • Upgrade cuDNN to 9.15.1 for CUDA 13 builds (#169412)
Distributed

  • c10d

    • Added handling of discontiguous allgather/reducescatter inputs (#163712)
    • Supported high stream for ProcessGroupXCCL (#163049)

  • Context Parallel

    • Introduced ContextParallal plan for parallelize_module (#162542)
    • Replaced context_parallel context manager with functional APIs (#164500)
    • Introduced flex_cp_forward custom op for FlexAttention CP (#163185)
    • Add _templated_ring_attention to the backward compatility stub (#166991)
    • Added _LoadBalancer classes, and load-balance interface to Context Parallel APIs with process-time based Round-Robin load-balance (#161062, #163617)
    • Added python bindings for NCCL CTA policies (#164309)

  • DeviceMesh

  • FullyShardDataParallel (FSDP1 and FSDP2)

    • Implemented idempotent reset_sharded_param: no-op if _local_tensor is already padded (#163130)
    • Added support of AC(FSDP) for torchtitan's MOE (#164009)
    • Provided public API to share cuda streams across roots (#165024)

  • DTensor

  • SymmetricMemory

    • Added MemPool support to CUDA backend and get_mem_pool API (#169740, #170008, #169739)
    • Added op multimem_one_shot_reduce_out (#164517)
    • Added op multi_root_tile_reduce (#162243, #164757)
    • Added op to get remote tensors (#167779)
    • Added symm_mem_sync Triton kernel to torch.ops.symm_mem (#168917)
    • Added a NVSHMEM based one side API (#159837, #163194)
    • Skipped multicast initialization if it fails (#163750)
    • Supported copy engine based all-gather and all-to-all (#170344, #170265)
    • Added set_signal_pad_size API for SymmetricMemory (#169156)

  • Pipeline Parallelism

    • Made runtime dbg log print custom actions (#167113)
    • Moved profiler record_function in schedule and improved visualizer (#164976, #160474)
    • Enabled inspect of schedule IR with comms (#162996)
    • Use default export mode (non-strict) for pipeline parallelism (#164045)
    • Enabled PP split BlockMask into micro-BlockMask (#164111)
    • Migrate other schedules to use PipelineScheduleRuntime (#164777)
    • Improvement the composability with FSDP with FSDP reduce scatters moved to end of step and backward_counter updated to schedule class (#165106, #165513)
    • Added optional argument to not save outputs (#165822)
    • Added PP Runtime Features for supporting Graph Based execution (#167277)
    • Used same dtype for receive and send tensor when initializing p2p communication. (#165539)
    • Support OVERLAP_F_B in schedule (#161072)
    • Support custom callback functions in schedule (#162016)

  • torchelastic

    • Added support to handle IGUSR1 and SIGUSR2 in multiprocessing (#160690)
    • Captured exit codes after sigterm/sigkill from torch elastic. (#160908)
    • Duplicate stdout and stderr and apply custom filter in torchrun (#160712)
    • Added flush option to TailLog (#167169)
Dynamo

  • Turn on capture_scalar_outputs and capture_dynamic_output_shape_ops when fullgraph=True (#163121, #163123)

  • Improved tracing for dict key hashing (#169204)

  • Tracing support for torch.cuda.stream (#166472)

  • Improved tracing of torch.autograd.Functions (#166788)

  • Miscellaneous smaller tracing support additions:

  • Extend collections.defaultdict support with *args, **kwargs and custom default_factory (#166793)

  • Support for bitwise xor (#166065)

  • Support repr on user-defined objects (#167372)

  • Support new typing union syntax X | Y (#166599)

Export
  • Improved fake tensor leakage detection in export (#163516)
  • Improved support for tensor subclasses (#163770)
FX
  • Add tensor subclass printing support in fx/graph.py (#164403)
  • Update Node.is_impure check if subgraph contains impure ops (#166609, #167443)
  • Explicitly remove call_mod_node_to_replace after inlining the submodule in const_fold._inline_module` (#166871)
  • Add strict argument validation to Interpreter.boxed_run (#166784)
  • Use stable topological sort in fuse_by_partitions (#167397)
Inductor
  • Pruned failed compilations from Autotuning candidates (#162673)
  • Extend triton_mm auto-tune options for HIM shapes (#163273)
  • Various fixes for AOTI-FX backend
  • Solve for undefined symbols in dynamic input shapes (#163044)
  • Support symbol and dynamic scalar graph inputs and outputs (#163596)
  • Support unbacked symbol definitions (#163729)
  • Generalize FloorDiv conversion to handle more complex launch grids. (#163828)
  • Don't flatten constant args (#166144)
  • Support SymInt placeholder(#167757)
  • Support torch.cond (#163234)
  • Add tanh, exp, and sigmoid activations for Cutlass backend. (#162535) (#162536)
  • Hardened the experimental horizontal fusion torch._inductor.config.combo_kernels (#162442) (#166274) (#162759) (#167781) (#168127) (#168946) (#168109) (#164918)
  • Enable TMA store for TMA matmul templates on Triton. (#160480)
  • Add Blackwell GPU templates (persistent matmul, FP8 scaled persistent + TMA GEMMs, CuTeDSL grouped GEMM, FlexFlash forward, FlexAttention configs). (#162916) (#163147) (#167340) (#167040) (#165760)
  • Support qconv_pointwise.tensor and qconv2d_pointwise.binary_tensor quantized operations. (#166608)
  • Support out_dtype argument for matmul operations. (#163393)
  • Add support for bound methods in pattern matcher. (#167795)
  • Add way to register custom rules for graph partitioning. (#166458) (#163310)
  • Add codegen support for fast_tanhf on ROCm. (#162052)
  • Support deepseek-style FP8 scaling in Inductor. (#164404)
  • Enable int64 indexing in convolution and matmul templates. (#162506)
  • Add SDPA patterns for T5 variants when batch size is 1. (#163252)
  • Add mechanism to get optimal autotune decision for FlexAttention. (#165817)
  • Add fallback config fallback_embedding_bag_byte_unpack. (#163803)
  • Expose config for FX bucket all_reduces. (#167634)
  • Add in-kernel NaN check support. (#166008)
  • Enable pad_mm and decompose_mm_pass pass on Intel GPU. (#166618) (#166613)
  • Improve CUDA support for int8pack_mm weight-only quantization pattern. (#161680) (#161848) (#163461)
  • Improve heuristics for pointwise kernels on ROCm. (#163197)
  • Enable mix-order reduction fusion earlier and allow fusing more nodes. (#168209)
  • Make mix order reduction work with dynamic shapes (#168117)
  • Better use of memory tracking (#168121)
  • Turn on LOAF (for OSS) by default. (#162030)
  • Log kernel autotuning results to CSV. (#164191)
  • Add warning for CUDA graph re-recording from dynamic shapes. (#162696)
  • Quiesce triton compile workers by default. (#169485)
  • Support masked vectorization for tail loops with integer and bool datatypes. (#165885)
  • Support tile-wise (1x128) FP8 scaling in Inductor. (#165132)
  • Support fallback for all GEMM-like operations. (#165755)
  • Enable Triton kernels with unbacked inputs. (#164509)
  • Add AVX512-VNNI-based micro kernel for CPU GEMM template. (#166846)
  • Support mixed dtype in native_layer_norm_backward meta function. (#159830)
  • Add tech specs for MI350 GPU. (#166576)
  • Add assume_32bit_indexing inductor config option. (#167784)
  • Wire up mask_mod and blockmask to FlexFlash implementation. (#166359)
  • More aggressive mix order reduction for better fusion. (#166382)
  • Mix order reduction heuristics and tuning. (#166585)
  • CuteDSL flat indexer needs to be colexigraphic in coordinate space (#166657)
MPS
Nested Tensor (NJT)
  • Added NJT support for share_memory_ (#162272)
torch.nn
  • Support batch size 0 for flash attention in scaled_dot_product_attention (#166318)
  • Raise an error when using a sliced BlockMask in nn.functional.flex_attention (#164702)
ONNX
  • Improved graph capture logic to preserve dynamic shapes and improve conversion success rate
  • Cover all FX passes into backed size oblivious (#166151)
  • Set prefer_deferred_runtime_asserts_over_guards to True (#165820)
  • Various warning and error messages improvements (#162819, #163074, #166412, #166558, #166692)
  • Improved operator translation logic
  • Update weight tensor initialization in RMSNormalization (#166550)
  • Support enable_gqa when dropout is non-zero (#162771)
  • Implement tofile() in ONNX IR tensors for more efficient ONNX model serialization (#165195)
Optimizer
  • Make Adam, AdamW work with nonzero-dim Tensor betas (#149939)
Profiler
  • Expose Kineto event metadata in PyTorch Profiler events (#161624)
  • Add user_metadata display to memory visualizer (#165939)
  • Add warning for clearing profiler events at the end of each cycle (#168066)
Python Frontend
  • Improved torch.library and custom ops to support view functions (#164520)
  • Rework PyObject preservation to make it thread safe, significantly simpler and better handle some edge cases (#167564)
  • Remove reference cycle in torch.save to improve memory usage (#165204)
  • Add generator arg to rand*_like APIs (#166160)
  • support negative index arguments to torch.take_along_dim negative (#152161)
Quantization
  • half and bf16 support for fused_moving_avg_obs_fake_quant (#162620, #164175)
  • bf16 support for fake_quantize_learnable_per_channel_affine (#165098)
  • bf16 support for backward of torch._fake_quantize_learnable_per_tensor_affine (#165362)
  • Add NVFP4 two-level scaling to scaled_mm (#165774)
  • Add support for fp8_input/fp8_weight/bf16_bias and bf16_output for fp8 qconv in CPU (#167611)
  • Make the torch.float4_e2m1fn_x2 dtype support equality comparisons (#169575)
  • add copy_ support for torch.float4_e2m1fn_x2 dtype (#169595)
Release Engineering
ROCm
  • Allow custom OpenBLAS library name for CMake build (#166333)
  • Add gfx1150 gfx1151 to binary build targets (#164782, #164854, #164763)
  • hipSPARSELt support - Update cuda_to_hip_mappings.py (#167335)
  • New implementation of upsample_bilinear2d_backward (#164572)
  • Remove env var HIPBLASLT_ALLOW_TF32 from codebase, TF32 always allowed (#162998)
  • Enable multi-arch compilation and unit tests for AOT Inductor (#166357)
  • Fix miopen batchnorm changing output format (#162112)
  • [ROCm] Enable multi-arch compilation and unit tests for AOT Inductor ([#166357](https://github.com/pytorch/pytorch/- pull/166357))
  • [ROCm][inductor] autotune support for persistent reduction kernels ([#163908](https://github.com/pytorch/pytorch/- pull/163908))
Sparse Frontend
XPU
  • Support --nproc-per-node torchrun option for Intel GPU (#159474)
  • Support complex dtype of Aten operator Matmul for Intel GPU (#160867)
  • Add SYCL-TLA implementation for aten flash attention (#169101)

Bug Fixes

Autograd
  • Fix custom autograd Function memory leak when saving mutated view (#164407)
  • Fix unused gradient tracking to respect create_graph (#168295)
  • Fix NaN gradients in atan2_backward when both inputs are zero (#166787)
  • Bugfix to forward autodiff causing different datatype 2 (#165784)
Build Frontend
C++ Frontend
  • Fixed C++ extension distributed warning spew (#162764)
CPU
CUDA
  • Handle python floats as double in CUDA C++ (#162626)
  • Use libnvrtc.so path based on CUDA version used by torch (#163642)
  • Handle python floats as double in CUDA C++ (#162626)
  • Use libnvrtc.so path based on CUDA version used by torch (#163642)
  • Fix torch.nonzero_static crash on CUDA when the input is a empty tensor (#162578)
  • Fix caller source location in C10_CUDA_CHECK error messages (#162808)
  • Fix channels-last dimension mapping in CUDA parallel_cat (#165023)
  • 64-bit indexing on CUDA:
    • Fix a large tensor indexding crash (#164049)
    • Handle 64-bit outer dimension in cumsum (#167326)
    • Fix crash in embedding_dense_backward (#165095)
    • Fix chunk-size for 64-bit indexing in fused adagrad (#165971)
  • Remove erroneous const_cast in CUDA memcpy call (#168165)
  • Handle large shared memory in torch.cuda._compile_kernel (#162647)
  • Fix torch.unique_consecutive crash on CUDA (#162950)
  • Fix correctness of parallel_cat (#165446)
  • Fix race condition and make torch.kthvalue deterministic on CUDA (#165762)
  • Fix shared memory race in reduce_kernel (#162995)
  • Fix Tensor.__dlpack__(stream=None) support during CUDA Graph capture (#163242)
  • Remove explicit casting of complex nansum during accumulation to avoid precision loss (#165494)
  • Disable jiterator for complex tan and tanh due to numerical issues (#165250)
  • Fix a few issues with out_dtype overload for addmm/baddbmm (#167931)
  • Fix safety issues when calling cuBLAS from multiple threads (#167248)
cuDNN
  • Disable cuDNN for 3D convolutions with kernel size != 1 for cuDNN 9.8+ due to a numerical isssue (#163581)
Dataloader Frontend
  • Fix pin memory return type when input is a tuple (#169690)
Distributed

  • c10d

    • Enforced P2P tensors to be dense (#163719)
    • Fixed split_group bug by having the parent pg option deep copied (#167125)
    • Fixed ProcessGroupNCCL coalseced profiling (#160680)

  • Context Parallel

    • Fixed cuDNN Context Parallel LSE dimension bug (#163231)

  • DistributedDataParallel: (DDP)

    • Fixed complex datatype handling in ddp (#166863)

  • DistributedStateDict

    • Fixed keyerror when loading parameter with unsaved optimizer state (#165228)

  • DTensor

  • FullyShardDataParallel (FSDP1 and FSDP2)

    • Added skipping reduce_scatter when world size is 1 (Collectives) (#162021)
    • Used grad div factor when fsdp_degree=1 (#167178)

  • Pipeline Parallelism:

    • Fixed FSDP unshard/reshard (#164775)
    • Fixed pipeline parallelism not correctly initializing backwards stages when evaluating before training. (#162823)
    • Fixed split_args_kwargs_into_chunks issues (#165306)
    • Fixed edge case with FSDP when stages_per_rank > 3 (#165467)

  • SymmetricMemory

    • Fixed memory allocation hold-up (#162680)
Distributed Checkpointing
  • Avoid multiple storage writer resets in async save (#159448)
  • DTensor slice dequantization with proper block alignment (#163532)
  • Add option to use PrefixStore to create checkpoint background process (#166560)
Dynamo
  • Fixed cProfile usage with torch.compile in Python 3.12+ (#170013)
  • Fix memory leak in tensor subclass metadata guard (#167352)
FX
  • Fix splitter for empty subgraph case (#161716)
  • Use tuples to have a deterministic ordering in shape prop. (#164851)
Inductor
  • Fix some edge cases (#162295)
  • Fix TMA transpose logic to handle 1D shapes + string differences (#163966)
  • fix flex attention eager: dont round down scores to low-precision (closes #163588) (#163986)
  • Fix a condition error in torch/_inductor/codegen/debug_utils.py (#165033)
  • Thread deterministic config vars to subproc compilation (#165729)
  • Fix identity expansion. (#165066)
  • Fix FP8 activation quantization for duplicate forward outputs. (#163364)
  • Fix decomposition issues (repeat_interleave out-of-bounds indices, divmod error, alpha/beta handling). (#165368) (#163482) (#167317)
  • Fix dynamic shaped heads check in FlexFlash. (#165866)
  • Fix argmin/argmax returning incorrect indices for non-contiguous tensors. (#165983)
  • Fix unbacked float symbol handling in kernel codegen. (#166890)
  • Fix static_input_indices subclass remapping under training. (#167127)
  • Fix torch.cond HOP device handling in inductor. (#167354)
  • Fix CppTile2DKernel for FP8 datatype. (#167451)
  • Fix user-defined Triton kernel output with .cpu() correctness issue. (#168281)
  • Fix viewed outputs getting padded incorrectly. (#163398)
  • Fix lowering issues (as_strided with .view(dtype) inputs, symbolic shapes in FlexAttention, sym_size_/sym_stride). (#163319) (#168383) (#167565)
  • Fix error from custom CUDA allocators. (#163422)
  • Fix copy_ for scalar in inductor. (#164167)
  • Fix bug with serialization after AOTAutogradCache hit. (#165474)
  • Fix searchsorted for non-dense tensors. (#165064)
  • Fix constant folder issues. (#166655)
  • Fix constant creation issues. (#167398)
  • Fix picking wrong contiguous node. (#168371)
  • Fix inner reduction decision logic. (#168391)
  • Fix device determination logic in Conditional. (#169199)
  • Fix pattern matcher FailedMatch format string. (#169611)
  • Fix SyntaxError from truncated Unicode escape in Windows compile-time auto-tuning block. (#169286)
  • Optimize sum reduction heuristics. (#163144)
  • Optimize scalar welford_reduce. (#162709)
  • Disable mixed-order reduction for cpp-wrapper. (#169859)
  • Capture Triton timeout errors without crashing the job. (#169064)
  • Correctly set max_numwarps in coordinate descent tuner. (#159146)
  • Fix Triton group_m config. (#169514)
  • Fix convolution autotune check when groups != 1. (#163094)
  • Fix constant shape for float constants. (#164241)
  • Fix Diode/exhaustive autotune crash on AMD. (#169225)
  • Fix get_raw_stream undefined error. (#163707)
  • Fix runtime error in context on cpu-only machine when compile for GPU. (#165220)
  • Fix AMD CPU max-autotune breakage. (#168079)
  • Fix bad merge duplicate pre pass. (#165917)
  • Fix layout constraint for weight_norm backward. (#167667)
  • Fix cross-process group overlap. (#169384)
  • Fix WeakDeps (WAR deps) handling during fusion. (#162316)
  • Fix unbacked replacement where LHS is purely backed expr and RHS is unbacked expr. (#164013)
  • Fix stride rounding on Blockwise128x128 to accommodate for small shapes. (#164953)
  • Fix loop pipelining for 2D/2D case of Triton grouped MM. (#165265)
  • Fix persistent rblock statically_known_leq error cases. (#165657)
  • Fix bugs in emulate_precision_casts (#163520)
  • Support python slicing with tensor inputs. (#165074)
Ahead-Of-Time Inductor (AOTI)
  • Bugfix for doing negative padding (#161639)
  • Fix unbounded number of substitutions when equality checks contain Max expr (#163685)
  • Use atomic API when trying to apply size hints to input tensor strides. (#163660)
  • Fix a mixed-device bug for scatter_add (#167341)
  • Fix a small buffer mutation issue (#169347)
  • Fix aot_compile typing. (#168320)
MPS
Nested Tensor (NJT)
  • Fixed NJT min / max operations on integer dtypes (#162273)
torch.nn
  • Fix silent correctness when backpropagating to score_mod in nn.functional.flex_attention (#163677)
  • Fix bug in nn.Module.load_state_dict for singleton tensor (#166335)
ONNX
  • Native ONNX ops (torch.onnx.ops)
  • Fix rotary_embedding_23 implementation (#162865)
  • Create fake implementations for onnx ops; fix boolean mask in attention (#165780)
  • Fix onnx export on big endian machines (#167816)
Optimizer
  • Fix SWALR.state_dict and load_state_dict to serialize properly with weights_only=True (#163122)
  • Prevent problematic tensor aliasing in LRScheduler (#163098, #163120)
  • Fix LBFGS wolfe max iteration (#161488)
Profiler
  • Fix ProfilerState typo ('Disable' → 'Disabled') and expose PRIVATEUSE1 in ActiveProfilerType (#169166)
ROCm
  • Fix hardsigmoid op (#162758)
  • Fix GEMM carveout feature (#164303)
  • Disable __builtin_amdgcn_rcpf for gfx90a (#166454)
  • ROCm 7.0 BC-breaking preparations in JIT support (#160587, #166147)
Sparse Frontend
  • Fix mul(COO, COO) on MPS for hybrid COO variants (#166164)
  • Update torch.sparse_coo_tensor error message to include more information about input tensor properties (#161900)
  • Fix GradTrackingTensor sparse layout propagation (#165765)
XPU
  • Fix OneDNN deconvolution with output_padding on Intel GPU (#169176)
  • Fix conv1d precision error on Intel GPU (#162944)
  • Fix incorrect FLOPs counting of convolution_overrideable on Intel GPU(#166839)
  • Fix performance drop in AOTI on Intel GPU (#163315)

Performance

Benchmark
  • Add attention benchmarking numbers to pytorch operator microbenchmarks (#164155)
CPU (AArch64)
CUDA
  • Integrate NVIDIA cuSolver backend into ATen/Linalg (initial implementation for eig/eigval) (#166715)
  • Reduce register pressure in radix_sort_pairs to improve torch.sort performance (#167094)
  • Add Flash Attention 4 to sdpa (#167348)
  • Vectorize stores in cat for all dtypes on CUDA (#162440)
  • Expose pinned_reserve_segment_size_mb to speed up pinned memory allocation (#164501)
  • torch.topk: refactor global histogram/cumsum into a dedicated kernel to improve performance on CUDA (#164459)
  • Vectorize 8 elements on 16=bit data types for sum/mean to improve performance (#165055)
  • Switch order of blocked reduce when vectorize loads to improve performance (#165178)
  • Reduce register pressure to improve torch.EmbeddingBag performance (#167834)
  • Make clamp kernel branchless to improve performance (#167889)
cuDNN
  • Reenable cuDNN for 64-bit depthwise convolutions (#168364)
Distributed Checkpointing
  • Add timeout for checkpoint background process join (#162828)
  • Disable GC in process based async checkpointing (#169613)
  • Optimize global save-plan validation (#166820)
  • state dict staging fixes (#166025)
Dynamo
  • Faster tracing of some pytree functions (#168342)
FX
  • Move Node._prepend/Node._remove_from_list to C++ (#165882)
  • Optimize torch.fx.Node.replace_all_uses_with (#165889)
Inductor
  • Naive foreach autotune support (#162053)
  • Invert unary read and write for better fusion. (#161404)
  • Generate fused RMS/layer norm backward. (#165370)
  • Optimize cold compile time when cudagraphs-partition is enabled. (#167132)
  • Reduce cold compilation time caused by duplicated user-defined Triton kernels. (#168292)
  • Optimize identity permute in empty_permuted decomposition. (#169731)
  • Properly enlarge XBLOCK/set num_warps=1 for B200 inner persistent reductions. (#168335)
  • Improved heuristic for operator reordering for peak memory. (#161810)
  • Add more configs for pointwise kernels on ROCm. (#166470)
  • Improve A16W8 performance for CPU GEMM template. (#162479)
  • Make mix-order-reduction split size not depends on split-reduction heuristics (#166461)
  • Less aggressive persistent reduction when it could induce large masking with dynamic shapes. (#163365)
  • Improve FlexAttention backward configs for B200 (#163318)
Quantization
  • Make prepare and convert faster by caching (#162550)
  • Add onednn context cache for CPU qlinear to improve performance (#168150)
Release Engineering
  • Add operator microbenchmarks for attention, convolution, and optimizer operations to CI (#165915, #166331, #168101)
  • Add HuggingFace LLM benchmarks and cleanup benchmark model configurations (#156967, #164815, #164816)
ROCm
  • Use hipSolver instead of MAGMA for Cholesky (#163977)
  • Layer norm now uses __builtin_amdgcn_rcpf(x) instead of 1.f/x (#165589)
  • OffsetCalc Unroll Optimization (#161700)
  • Improve perf for elementwise broadcast with mixed dtype (#163562)
  • Implement float32 copy kernel (#163869)
  • Improve non stride-one backwards indexing for small index sets (#164409)
  • Adjust grid size for non-unit stride backwards indexing (#165026)
  • Normalization update to block size (#165941)
  • Deserialize loads in planer sum portion of reduce() of norm. (#165927)
  • Deserialize loads in planer sum portion of stats() of norm (#166021)
  • Specialized binary elementwise broadcast kernel for mixed dtypes with float/bfloat16/half (#167233)
  • Roll kernel as grid stride loop (#169474)
  • Inductor performance improvements via configs, heurstics, and/or codegen (#163908, #161280, #166470, #162052, #163197)
torch.func
  • 20x less memory use and 37.25% speedup in min_cut_rematerialization_partition when using the new dp knapsack solver, compared to existing default one (dp) (#160914)

Documentation

Autograd
  • Add inference_mode hint message to use eval with inference. (#163619)
CUDA
  • Add Documentation for Device APIs (#162834)
  • Adding aliases for CUDA and XPU API documentation (#162984)
  • Clarify safety of CUDA graph memory pool sharing across graphs in documentation (#166975)
Distributed
  • c10d
    • Complete documentations for all distributed c10d apis (#165194)
Dynamo
  • Updated documentation for tlparse (#171339). tlparse is a compilation report tool that processes TORCH_TRACE logs to generate interactive HTML reports showing how your model was compiled. When reporting bugs to PyTorch developers, we encourage you to attach the trace log or tlparse output to provide critical debugging information to help us bisect the issue.
FX
  • Add docs for torch.fx.experimental.unification (#167334)
  • Fix the split_module tutorial code (#166154)
Inductor
  • Updated documentation for tlparse (#171339) (#162975). tlparse is a compilation report tool that processes TORCH_TRACE logs to generate interactive HTML reports showing how your model was compiled. When reporting bugs to PyTorch developers, we encourage you to attach the trace log or tlparse output to provide critical debugging information to help us bisect the issue.
  • Update FlexConfig documentation. (#162533)
Ahead-Of-Time Inductor (AOTI)
  • [AOTI] Update AOTInductor tutorial (#163808)
torch.nn

  • Update CTCLoss docs float32 input required for CuDNN (#162042)

  • Update LPPool docs to clarify ceil_mode padding semantics when ceil_mode=True (#163186)

ONNX
  • Update export docstring (#162622)
  • Fix incorrect attention example in ONNX exporter docstring (#167646)
Profiler
  • Add documentation for FunctionEvent (#167688)
Quantization
  • Document some quantization public apis (#165160)
  • Add missing method docstrings for pytorch quantization classes (#165199)
XPU
  • Add new supported client GPU Panther Lake in "Get Started with XPU" page (#170517)

Security

Developers

Composability
  • Removed guard_size_oblivious from internal code, replacing most usages with guard_if_{false|true}. Both APIs are used in framework code that gets traced through to make it more friendly to unbacked symints, but the new APIs are more intuitive (#164664, #164665, #167232)
Distributed

  • c10d

  • DTensor

    • Added guide for what to do about mixed torch.Tensor and DTensor operations (#162651)
    • Raised an RuntimeError when checkpointing APIs are used with Partial placement (#163941)

  • torchelastic

    • Added missing signals_to_handle to launcher logging (#166631)
    • Added logging exit code for failures to ease debugging (#160907)
FX
  • Refactor proxy_tensor (#165266)
  • Fix invalid symbol definition emitted in fx_graph_runnable.py (#166529)
  • Add debug-level logging to Interpreter.run_node (#117351) (#166622)
  • Fix an unsafe indexing in fx exception handling (#169140)
  • Type annotations for torch/_higher_order_ops/flat_apply.py (#168933)
  • Add recompute tags (from AC) into GraphModule.print_readable() by default (#167735)
  • Apply ruff UP035 rule (#165214, #163744)
  • Add model code stack trace to cuda.memory._snapshot (#166676)
  • Add model code stack trace to torch.profile (#167110)
  • Move enrich_profiler_metadata config import out of gm.recompile() (#167114)
Inductor
  • Add API for scheduling overlap from inductor configs. (#169693)
  • Make LOCK_TIMEOUT in codecache configurable. (#165030)
  • Add debug output for specific pattern matching. (#169603)
  • Add overridable env var for disabling FX graph cache. (#166138)
  • Add subsystem support to pattern matcher. (#163922)
  • Add pre-grad graph bisecting support. (#166344)
  • Decouple flags for optimization and debug symbols. (#167385) (#167575)
  • Introduce HOP for inductor compiled regions to allow torch dispatch. (#167844)
  • Record triton kernels, run-to-run determinism checks (#167028)
Release Engineering
  • Migrate from setup.py to modern Python build tools (pip install and python -m build) (#156711, #156712)
ROCm
  • Add Rocm to Operator Microbenchmark CI (#164173)
  • Enable TD for all ROCm default and distributed config workflows (#168225)
  • Expand trunk.yml coverage for ROCm (#168162)
  • cudagraph trees ut fixes (#163592)
  • test_convolution.py uses miopen immediate mode (#164598)
  • Keep amdgpu-coerce-illegal-types flag if rocm version is less than 7.2 (#165789)
  • Use a ROCm version string without hash. (#166336)
  • Dynamo benchmarks: remove outdated flaky models and enable deterministic algorithms (#169024)
XPU
  • Upgrade Intel GPU software stack package to intel-deep-learning-essentials-2025.3 (#166829)
View originalPermalink
v2.9.1

PyTorch 2.9.1 Release, bug fix release

This release is meant to fix the following issues (regressions / silent correctness):

Tracked Regressions

Significant Memory Regression in F.conv3d with bfloat16 Inputs in PyTorch 2.9.0 (#166643) This release provides work around this issue. If you are impacted please install nvidia-cudnn package version 9.15+ from pypi. (#166480) (#167111)

Torch.compile

Fix Inductor bug when compiling Gemma (#165601) Fix InternalTorchDynamoError in bytecode_transformation (#166036) Fix silent correctness error_on_graph_break bug where non-empty checkpoint results in unwanted graph break resumption (#166586) Improve performance by avoiding recompilation with mark_static_address with cudagraphs (#162208) Improve performance by caching get_free_symbol_uses in torch inductor (#166338) Fix fix registration design for inductor graph partition for vLLM (#166458) (#165815) (#165514) Fix warning spamming in torch.compile (#166993) Fix exception related to uninitialized tracer_output variable (#163169) Fix crash in torch.bmm and torch.compile with PyTorch release 2.9.0 (#166457)

Other

Fix warning spamming on new APIs to control TF32 behavior (#166956) Fix distributed crash with non-contiguous gather inputs (#166181) Fix indexing on large tensor causes invalid configuration argument (#166974) Fix numeric issue in CUDNN_ATTENTION (#166912) (#166570) Fix symmetric memory issue with fused_scaled_matmul_reduce_scatter (#165086) Improve libtorch stable ABI documentation (#163899) Fix image display on pypi project description section (#166404)

View originalPermalink
v2.9.0

2.9 Release Notes

PyTorch 2.9.0 Release Notes

Highlights

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Backwards Incompatible Changes

Min supported Python version is now 3.10 (#162310)

The minimum version of Python required for PyTorch 2.9.0 is 3.10. We also have 3.14 and 3.14t available as preview with this release.

Undefined behavior when an output of a custom operator shares storage with an input

This is a reminder that outputs of PyTorch custom operators (that are registered using the torch.library or TORCH_LIBRARY APIs) are not allowed to return Tensors that share storage with input tensors. The violation of this condition leads to undefined behavior: sometimes the result will be correct, sometimes it will be garbage.

After #163227, custom operators that violated this condition that previously returned correct results under torch.compile may now return silently incorrect results under torch.compile. Because this is changing the behavior of undefined behavior, we do not consider this to be a bug, but we are still documenting it in this section as a "potentially unexpected behavior change".

This is one of the conditions checked for by torch.library.opcheck and is mentioned in The Custom Operators Manual

More details

Outputs of PyTorch custom operators are not allowed to return Tensors that share storage with input tensors

For example, the following two custom operators are not valid custom operators:

@torch.library.custom_op("mylib::foo", mutates_args=())
def foo(x: torch.Tensor) -> torch.Tensor:
    # the result of `foo` must not directly be an input to foo.
    return x

@torch.library.custom_op("mylib::bar", mutates_args=())
def bar(x: torch.Tensor) -> torch.Tensor:
    # the result of bar must not be a view of an input of bar
    return x.view(-1)

The easiest workaround is to add an extra .clone() to the outputs:

@torch.library.custom_op("mylib::foo", mutates_args=())
def foo(x: torch.Tensor) -> torch.Tensor:
    return x.clone()

@torch.library.custom_op("mylib::bar", mutates_args=())
def bar(x: torch.Tensor) -> torch.Tensor:
    return x.view(-1).clone()

A common way to get into this situation is for a user to want to create a custom operator that sometimes mutates the input in-place and sometimes returns a new Tensor, like in the following example.

@torch.library.custom_op("mylib::baz", mutates_args=["x"])
def baz(x: torch.Tensor) -> torch.Tensor:
    if inplace:
        x.sin_()
        return x
    else:
        return x.sin()

This dynamism is not supported and leads to undefined behavior. The workaround is to split the custom operator into two custom operators, one that always mutates the input in-place, and another that always returns a new Tensor.

@torch.library.custom_op("mylib::baz_outplace", mutates_args=())
def baz_outplace(x: torch.Tensor) -> torch.Tensor:
    return x.sin()

@torch.library.custom_op("mylib::baz_inplace", mutates_args=["x"])
def baz_inplace(x: torch.Tensor) -> torch.Tensor:
    x.sin_()

def baz(x):
    if inplace:
        baz_inplace(x)
        return x
    else:
        return baz_outplace(x)
Build metal kernels of MacOS-14+ and remove all pre-MacOS-14 specific logic, requires MacOS-14+ going forward (#159733, #159912)

PyTorch MPS is only supported on MacOS-14 or later. If you need to use MPS on MacOS Ventura, please avoid updating to Python-3.9 or above

Upgrade to DLPack 1.0 (#145000)

This upgrade is doing the same BC-breaking changes as the DLPack release. Objects in torch.utils.dlpack have been updated to reflect these changes, such as DLDeviceType.

See the PR for details on the exact changes and how to update your code.

Raise appropriate errors in torch.cat (#158249)

torch.cat now raises ValueError, IndexError or TypeError where appropriate instead of the generic RuntimeError. If you code was catching these errors, you can update to catch the new error type.

Default to dynamo=True for ONNX exporter (#159646, #162726)

Previously torch.onnx.export(...) used the legacy TorchScript exporter if no arguments were provied. The ONNX exporter now uses the newer torch.export.export pipeline by default (dynamo=True). This change improves graph fidelity and future-proofs exports, but may surface graph capture errors that were previously masked or handled differently.

Previously in torch 2.8.0:

# API calls the legacy exporter with dynamo=False
torch.onnx.export(...)

Now in torch 2.9.0:

# To preserve the original behavior
torch.onnx.export(..., dynamo=False)

# Export onnx model through torch.export.export
torch.onnx.export(...)

Recommendation: first try the new default; only fall back if you hit blocking issues and report them upstream. Long term solution: fix the root cause instead of relying on fallback or TorchScript exporter.

Switch off runtime asserts by default in Export in favor of a shape guards function (#160111, #161178, #161794)

To enable runtime asserts, use export(..., prefer_deferred_runtime_asserts_over_guards=True). Also kills the allow_complex_guards_as_runtime_asserts flag, merging it into the former option.

Additionally, exported_program.module() will generate a call to a _guards_fn submodule that will run additional checks on inputs. Users who do not want this behavior can either remove this call in the graph, or do exported_program.module(check_guards=False) to avoid the generation.

Set default opset to 20 in ONNX (#158802)

Opset 20 enables newer operator definitions. If your tooling or downstream runtime only supports opset 18, pin it explicitly. For the latest ONNX operators, you can experiment with opset 23.

Previously in torch 2.8.0:

# opset_version=18
torch.onnx.export(...)

Now in torch 2.9.0:

# To preserve the original behavior
torch.onnx.export(..., opset_version=18)

# New: opset_version=20
torch.onnx.export(...)

# Use the latest supported opset: opset_version=23
torch.onnx.export(..., opset_version=23)
Drop draft_export in exporter API (#161454, #162225)

Remove implicit draft tracing from the default exporter path, achieving clearer behaviour and faster failures. The expensive torch.export.draft_export diagnostic path is no longer auto-invoked (which could take hours on large models). You can still opt in for deep diagnostics:

Previously in torch 2.8.0:

# If both torch.export.export(..., strict=False) and
# torch.export.export(..., strict=True) fail to capture
# the model graph, torch.export.draft_export(...) will be triggered,
# and uses real tensor to trace/export the model.
#
# Inside export_to_onnx.py:
#  ... torch.onnx.export(..., dynamo=True)
python export_to_onnx.py

Now in torch 2.9.0:

# To trigger torch.export.draft_export once
# torch.export.export strict=False/True both
# fail:

TORCH_ONNX_ENABLE_DRAFT_EXPORT=True python export_to_onnx.py
Remove torch.onnx.dynamo_export and the onnxrt torch compile backend (#158130, #158258)

torch.onnx.dynamo_export is removed. Please use torch.onnx.export instead. The experimental ONNX Runtime compile backend (torch.compile(backend="onnxrt")) is no longer supported.

Remove torch.onnx.enable_fake_mode (#161222)

The dynamo=True mode uses FakeTensors by default which is memory efficient.

Some public facing ONNX utility APIs for the TorchScript based exporter are now private (#161323)

Deprecated members in torch.onnx.verification are removed. Previously private torch.onnx.symbolic_opsets* functions will no longer be accessible. Consider making a copy of the source code if you need to access any private functions for compatibility with the TorchScript based exporter.

Remove torch.onnx.symbolic_caffe2 (#157102)

Support for caffe2 in the ONNX exporter has ended and is removed.

Remove /d2implyavx512upperregs flag that slows build (#159431)

Re-introduced AVX512 optimizations for Windows VS2022 builds, may cause issues with specific versions of VS2022, see #145702

Add ScalarType to shim conversion and stable::Tensor.scalar_type (#160557)

Before, user extensions could only in abstract pass around obfuscated dtypes appearing as int32_ts. Now, users can confidently use torch::headeronly::ScalarType in their extensions for major scalar types. This PR enables ABI stability by adding a translation layer through the shim, so that even if the ScalarType enum values change in the future, user extensions need not fear.

This change adds ScalarType support for user extensions and is only narrowly BC breaking for unpopular dtypes: quint*s, qint*s, Bits*, dummy_uint*s, dummy_int*s, Float8_e8m0fnu, and Float4_e2m1fn_x2 in the use case where an extension retrieves a Tensor dtype of the above and passes it into aoti_torch_call_dispatcher.

Deprecations

Deprecate pin_memory_device param in torch.utils.data.DataLoader (#158323)

We move enabling pin_memory back inside BaseDataLoaderIter. This is required for StatefulDataloader which leveraged BaseDataLoaderIter direclty rather than the Dataloader class init

Deprecate torch.export.export_for_training API in favor of equivalent torch.export.export API (#158203)

torch.export.export_for_training exists because we couldn't migrate internal usages of export to the final IR. Now that we have completed the migration, we deprecated and deleted this API.

New Features

Python Frontend
  • Add utility to get the kernel currently registered on the dispatcher (#158393)
  • Extend __torch_function__ handler to be triggered by elements within a list (#160256)
  • Add torch.hash_tensor reduction function (#154149)
FX
  • Extend torch function support to ALL arguments instead of just scalar type (but not inside of list, #145089)
  • Add is_fx_symbolic_tracing flag (#161385)
Dynamo
  • Experimental API for ahead-of-time compiling models in fullgraph mode (#161383)
  • Add a hook for recompilations (#157961)
  • DynamicInts prototype (#162194)

Introduces an API for annotating dynamic integer inputs & attributes for torch.compile, by wrapping plain ints with DynamicInt(). DynamicInt objects also work in eager mode, acting as their underlying values when passed as scalar inputs.

a = DynamicInt(4)
y = a + 2  # DynamicInt(6)
z = torch.ones(a)  # torch.ones(4)

fn = torch.compile(torch.ones)
fn(a)  # compiled fn takes a dynamic integer input
fn(2)  # returns torch.ones(2) without recompiling
Optimizer
  • Introduce Muon optimizer to PyTorch (#160213)
Profiler
  • Add GC Events to Python Stack Tracer (#161209)
  • Add a custom profiler configuration option (#151656)
Inductor
  • Allow user to pass in custom partitioner function (#157580)
Export
  • Add support for param mutation under inference mode (#159661)
AOTDispatcher
  • Add AOTDispatcher config to set backward autocast behavior (#156356)
Quantization
  • Enable cpu fp8 qlinear and cpu fp8 qconv (#155678, #157076)
ONNX
  • RMS Norm support in opset 23 (#159377)
C++ Extensions
  • Build out a stable set of ATen ops in torch/csrc/stable/ops.h: amax, narrow, new_empty + new_zeros dtype variant, pad, (#159328, #158974, #159508, #161597, #160214)
  • Add torch::stable::Tensor() default constructor, is_cpu, and get_device_index(#159507, #160212, #160143)
  • Add beginnings of torch::stable::accelerator with support for DeviceGuard and Stream (#159679, #160453)
  • Start building out torch/headeronly: c10 Macros, STD_TORCH_CHECK, ScalarTypes (like BFloat16 and Half, #158035, #158365, #157912, #158377, #159302, #159414, #159412, #159415, #159411, #159911)
  • Remove cmake cache and reconfigure again if it is invalid (#156958)
  • Cut a version of TORCH_ERROR_CODE_CHECK in headeronly from AOTI (#159604)
  • Remove wheel from build requirements (#158027)
  • Error when TORCH_STABLE_ONLY is defined in TensorBase.h (#161658)
Build Frontend
  • Add transpose to torch/csrc/stable (#158160)
  • Add zero_() and empty_like(t) to torch/csrc/stable/ops.h (#158866)
Release Engineering

  • Add support for CUDA 13.0 in CI/CD builds. Enable CUDA compression mode for binary size reduction for CUDA 13.0 builds (#160956, #161073, #161257, #161663, #161316, #160201, #160770, #161013, #161916, #162268, #162322, #162383, #161833)

  • Enable CUDA 12.6, 12.8 and 13.0 support for Linux ARM64 CD builds (#162364, #160720, #159481)

  • Add support for Python 3.14 in CI/CD builds (#156889, #157559, #159261, #159869, #160593, #160788, #161255, #159725)

  • Enable NVSHMEM integration (#151261, #153010, #154538, #155506, #156685, #158938, #161321, #160778, #159907, #160465)

CUDA
  • Add getter for CUDA graph exec to allow mutation of captured kernel params (#161294)
  • Implement support for cudnn_batch_norm_out kernel to replace the autogen approach (#123020)
CPU
  • Support GQA for flash attention (#157893)
MPS
  • Partial sparse support for MPS backend (#159729, #160254, #160223, #161846, #162007, #157238)
  • Add avg_pool3d, max_unpool1d/2d/3d, max_pool3d, max_pool3d bwd pass, and avg_pool3d bwd pass for MPS (#158877,#159789, #156467, #157498, #159089)
ROCm
  • OCP Micro-scaling Format (mx-fp8/mx-fp4) Support (#151360)
XPU
  • Enable FlexAttention on Intel GPU (#143553)

Improvements

Python Frontend
  • Speed up torch.load under FakeTensorMode by reducing random reads (#157931)
  • Make torch.utils.benchmark.utils.timer accelerator agnostic (#157131)
  • Improve error message for weight-only load errors (#159935)
torch.nn
  • Allow register_buffer with Tensor-like objects (#159455)
  • Improve error message for unsupported padding configurations (#160866)
  • Validate target is 0D when input is 1D in NLLLoss (#161412)
Optimizer
  • Resolve warning in LBFGS when converting a tensor with requires_grad=True to a scalar (#160389)
  • Resolve SequentialLR deprecation warning about invoking step(epoch) (#149392)
Autograd
  • Support deterministic torch.nn.Upsample mode="trilinear" backward (#154239)
Distributed
c10d
  • Add improvements to eager init of ProcessGroupNCCL (#156748)
  • Simplify unique hash management of ProcessGroupNCCL (#156790)
  • Support per operation timeouts in ProcessGroupGloo (#158128)
  • Allow ping to be retried in TCPStore (#159165)
  • Support scalar tensor for functional all_gather (#149913)
  • Expos unsafe_get_ptr for dist.ProcessGroupNCCL.NCCLConfig (#161136)
  • Add batch option for send/recv_object_list (#160342)
  • Make FakeStore optional to be passed into fake backend (#162164)
  • Enable complex datatype support in ProcessGroupGloo (#156633)
  • Move thread-local capture mode guard to include work.isStarted (#160398)
DistributedDataParallel (DDP)
  • Support ddp zero hook XCCL path (#159240)
DTensor
  • Relax device_mesh argument constraint in local_map (#157049)
  • Support complex numbers in DTensor redistribute (#157329)
  • Rework partial propagation in point-wise op and support mul (#157340)
  • Allow dynamic shapes for DTensor slice (#157953)
  • Implement histc op (#158298)
  • Made dispatch to sharding prop over decomps (#159324)
  • Support user-supplied Generator for random ops (#159933)
  • Add propagate_tensor_meta function that skips cache if _are_we_tracing (#161334)
  • Support local_map as a decorator (#161353)
Device Mesh
  • Enable the use of user set backend and pg option even for the global mesh (#157501)
  • Enable slicing a submesh with warnings (#158899)
  • Allow controlling PG backend and options via init_device_mesh (#159371)
FullyShardedDataParallel2 (FSDP2)
  • Support custom all_gather and reduce_scatter comms (#155189)
  • Made it fail set_allocate_memory_from_process_group if used together with custom comm hooks (#157487)
  • Use reduceOpSum when world size is 1 (#157529)
  • Skipp allgather when world size is 1 (#160135)
  • Use post_reduce_stream.record_event() on hsdp+cpuoffload (#160481)
Tensor Parallel (TP)
  • Improve parallelize_module API to support more cases (#157182)
TensorPipe
  • Update TensorPipe pinned dependency version (#159834)
TorchElastic
  • Enable NUMA binding integration with elastic agent and torchrun (#149334)
  • Support NUMA Binding for Callable Entrypoints (#160163, #161183)
Pipeline Parallelism (PP)
  • Add eval() API to schedule (#157795)
  • Allow intermediate nodes in zero bubble to have multiple grads (#159084)
  • Support OVERLAP_F_B computation type (#158978)
  • Initializ P2P communicators on first step (#160210)
  • Add DualPipeV schedule (#159591)
Linear Algebra Frontend
  • Use rocSOLVER for Cholesky inversion on AMD. (#157154)
  • Add option for using TF32 as fp32 internal precision for matmul/linear/conv on MKLDNN (#157520)
  • Make einsum produce contiguous outputs in more cases (#161755)
Profiler
  • Add more CUDA API for kernel launcher (#156016)
  • Allow Custom Time Unit When Printing Profiler Table (#157913)
  • Update CUDA runtime kernel identification logic (#157890)
FX
  • Fix DCE eliminating random operations by improving is_impure() (#151524, #157981)
  • Support converting a float32 tensor to a scalar in FX trace. (#158216)
  • Correctly copy self.module_stack in ModuleStackTracer (#159956)
  • Add tool to track events in graph split (#159795)
  • Add node_name_match to subgraph rewriter (#157574)
Dynamo
  • Improve tracing support for various Python builtin data structures/modules:
    • lists (e.g. #153969)
    • sets (e.g. #153150)
    • dicts (e.g. #154794)
    • iter (e.g. #156371)
    • itertools (e.g. #159693)
    • collections (e.g. #159365)
    • collections.NamedTuple (#159367)
    • frozen dataclasses.dataclass (#159529)
  • Graph break error messages link to a website with more information (#159011)
  • Add option for TorchDispatchMode to ignore torch.compile internals (#161648)
Inductor
  • Add Inductor support for MTIA backend (#159211)
  • Share default device context when all graph partitions and cudagraph-unsafe ops are on the same device(#162873)
Ahead-Of-Time Inductor (AOTI)
  • Enable AOTI for CPU on Windows (#158915)
  • Re-enable TMA templates w/ AOTI (#157819)
  • Don't allow int32 indices if {non-inf, > int32_max} upper bound is provided (#159433)
  • Add RecordFunction to C shim so that profiling works with AOTI (#159842)
  • Add AOTI C shim functions for collective ops (#154492)
  • Add missing ops to set of C-shim ops which can have nullptr returns (#158073)
Export
  • Handle None & ellipsis slicing/select in non-strict (#157821)
  • Extend FP8 types in serialization (#158430)
  • Improve error messages for deserialization (#159881)
  • Support serialization for triton_kernel_wrapper_functional HOP (#161314)
  • Support serialization for complex constants (#161517)
  • Add runtime asserts to while_loop HOP subgraphs (#158467)
  • Warn on side-effectful code in strict mode (#160060)
  • Support for vmap in pre-dispatch export (#154650)
  • Support vmap and custom autograd function/improve DTensor constructor inefficiency (#162240)
AOTDispatcher
  • Skip logging in fp8 activation quantization if there are no nodes to be quantized (#158129)
  • Add aot_export_joint_with_descriptors and aot_compile_joint_with_descriptors (#158715)
  • Extract out prepare_aot_module_simplified for use in next PR (#158319)
  • Rename modules in AOTAutograd (#158449)
  • Track descriptors for all inputs/outputs of AOTAutograd traced graph (#158624)
  • Improve graph output alias with subclass error message (#159619)
  • Pass fw/bw compilers to aot_export_joint_with_descriptors (#159814)
Composability
  • Meta implementation for aten.add.Scalar (#161332)
  • aten.expand_copy decomp (#161688)
  • Fix result dtype cast in decomp for aten.linalg_vector_norm (#155111)
  • Add dtype checks in meta implementation for several ordering ops (#159556)
  • Fix meta function for aten.complex (#160894)
  • Improve unbacked symint (dynamic shape) support for several decompositions (#148815, #156902, #157008, #158894, #159184, #160683, #160253, #162084, #162099, #162109, #160462)
Quantization
  • Avoid getting model device once per node for pt2e quantization flow (#159901)
  • Fixes bug in implementation of HistogramObserver (#156457)
  • Support bias=None for fbgemm_linear_fp16_weight CPU op (#158535)
  • Add Static Dispatch Kernel for wrapped_fbgemm_linear_fp16_weight for Sigmoid (#160451)
Nested Tensor (NJT)
  • Added initial log_softmax() support (#159662)
Foreach
  • Invoke vector.reserve() consistently for non-inplace foreach operations (#161128)
  • Faster and safer lambda expression capture in has_integral_tensor() (#161042)
ONNX
  • Support symbolic arguments in ONNX exporter (#157734)
  • Fix torch.tensor warning in ONNX symbolic_opset10 export (#158835)
C++ Frontend
  • Generalized AllocatorConfig to be device-agnostic via new AcceleratorAllocatorConfig (#149601, #150312)
  • Added Scalar::isUnsigned() method (#159877)
  • Exposed ModelRunner from nativert as public (#159989)
  • Improve error message for torch.binomial enforcing float inputs (#157658)
Build Frontend
  • Fix dev warning in Dependencies.cmake (#159702)
  • Fix building system gloo with CUDA/HIP (#146637)
  • Build libtorch without NVSHMEM (#160910)
  • Improve BLAS feature detection (#143846)
Release Engineering
  • Enable vLLM testing workflow (#160583, #161565, #162292, #162000, #161797)
  • Enable Windows ARM64 CI testing (#148753, #161504)
  • Enable PyTorch ROCm CI for MI355X testing. (#158889)
CUDA
  • Make cublaslt/hipblaslt workspaces persistent (#156495)
  • Remove unnecessary warnings during the ATen compilation process (#157703)
  • Slightly improve error message from repeat_interleave kernel (#157996)
  • Add framework for explanations for common CUDA errors (#158395)
  • Upgrade KernelLauncher kernelLaunchCheck to print help string (#158896)
  • Prep for cutlass upgrade by ignoring Wunused-but-set-variable (#159276)
  • Workaround ATen SFINAE under libc++ (#161101)
  • Implement changes to CCCL (CUB/Thrust/LibCUDACXX) usage in ATen (#153373)
  • Add maybe unused flag to remove warning (#157655)
  • Use new CCCL API in v2.8 (#160554)
  • Improve cupy device placement when device is provided with explicit index (#158529)
CPU (AArch64)
  • Made PyTorch compilable with gcc-14 on ARM (#157867)
MPS
  • Add shifted_chebyshev_polynomial_[tuvw], igamma/igammac,grid_sampler_3d, native_dropout/native_dropout_backward (#157488, #161927, #160541, #162108)
  • Extend atomic operations to all int types (#158179)
  • Extend index_put to complex types (#160159)
  • Extend addmm to integral types (#160270)
  • Add support for unsigned types (#159094)
  • Add API to query GPU core count (#160414)
  • Add kthvalue (#161817)
  • Type-promote tensor-iterator common dtype (#160334)
  • Implement logcumsumexp metal kernel (#156858)
  • Enable dlpack integration (#158888)
  • Dynamic reductions (#159355)
  • Update avg_pool2d to use Metal kernel when ceil_mode=True (#161011)
ROCm
  • Additional hipify mappings (#158056, #158352, #161992)
  • Refactor composable_kernel (CK) backend user interface to improve user experience (#152951)
  • Allow use of rocSOLVER for Cholesky inversion. (#157154)
  • AOT Inductor enable gfx950 for max autotune using CK (#159195)
  • Add flag torch.backends.miopen.immediate to toggle MIOpen Immediate Mode instead of relying on deterministic=True and benchmark=False (#158951)
  • MIOpen convolutions no longer call reshape_ or unexpectedly change memory formats (#161687)
XPU
  • Support Intel GPU quantization ops in AOTInductor (#156572)
  • Add device_id to Intel GPU properties to distinguish iGPUs with identical names (#156481)

Bug Fixes

Python Frontend
  • Add option in torch.utils.cpp_extension.load_inline to override gencode (#156850)
  • Fix max_width computation in Tensor printing (#126859)
  • Improve pin_memory error message on CPU-only systems (#159994)
  • Making batching rule for F.embedding DTensor-aware (#162117)
Autograd
  • Fix torch.autograd.Function memory leak due to torch.utils.checkpiont early stopping (#161171)
  • Fix torch.autograd.graph.GradientEdge for torch.autograd.Function (#160098)
  • Match 0-dim gradients device type regardless of subclass-ness (#160165)
Distributed
c10d
  • Fix slow init due to repeated dns resolution failure in socket (#159596)
  • Fix setGroupName and setGroupDesc in group_split and merge_remote_group (#159429)
  • Fix a bug of distributed 'gather' with noncontiguous tensors on the Gloo backend (#158903)
  • Fix a bug of distributed 'gather' with noncontiguous tensors on the NCCL backend (#159549)
  • Fix data inconsistencies when using batch_isend_irecv with 2D tensor views by making P2P tensors dense (#163719)
  • Handle discontiguous allgather/reducescatter inputs (#163712)
Device Mesh
  • Fix the not incorrectly chained each of the strings as iterables (#160709)
DistributedDataParallel (DDP)
  • Fix incorrect interaction between DDPOptimizer and donated buffers (#160745)
DTensor
  • Fix DTensor handling of conjugate bit (#158030)
  • Fix OpSchema equality check (#161231)
  • Fix grouped_mm strategy for invalid stride cases (#158245)
  • Fix F.one_hot in DTensor (#162307)
  • Always disabled ShardingPropagation cache if compiling (#156868)
FullyShardedDataParallel (FSDP)
  • Fix the bug in FSDP offload pin_memory (#157147)
  • Fix to ensure writeback handles NO_SHARD correctly by flattening tensors before copying (#154369)
FullyShardedDataParallel2 (FSDP2)
  • Fix error message for fsdp_pre_all_gather (#160817)
  • Fix the issue with set_reduce_scatter_divide_factor errors and MixedPrecisionPolicy (#155964)
Pipeline Parallelism (PP)
  • Fix eval step under no_grad() (#159293)
  • Fix zero bubble schedules for eval() (#159475)
TensorPipe
  • Fix import torch if compiled without TensorPipe (#159461)
TorchElastic
  • Fix wrong log file name in the docs of torch.distributed.elastic.multiprocessing.start_processes() (#160396)
Linear Algebra Frontend
  • Avoid downcasts for fp16 matmul on the BLAS backend (#161999)
Profiler
  • Fix Linter for Global Annotations flag in Snapshot (#157858)
FX
  • Fix split_module with symint (#160093)
  • Fix getattr_recursive with ModuleList (#161204)
  • Skip const folding with symbolic expression (#161437)
  • Fix qualified name for methods of torch.Tensor (#162224)
Dynamo
  • Fix segfault due to interaction between Dynamo backends and torch.compiler.reset() (#156527)
  • Fix crash due to bad interaction with recompilations and with blocks in Python 3.11+ (#162318)
torch.nn
  • Fix silent correctness w/ backpropping grads for FlexAttention (#163677)
  • Fix return_lse warning message in FlexAttention (#163578)
  • Fix FlexAttention head broadcast (#163426)
Inductor
  • Fix wrong meta function for constant_pad_nd (#159878)
  • Fix learnable bias assertion error in Inductor (#161170)
  • Fix int64 from MutationOutput Buffer (#162020)
  • Fix Inductor CUDA sort NaN behavior (#159308)
  • Fix layout for local buf in outer loop fusion (#160857)
  • Fix slice scatter dtype consistency (#160851)
  • Fix 3d tiled online softmax (#162341)
  • Fix unsafe collective reorder past wait in Inductor (#157489)
  • Fix FallbackKernel alias function to avoid incorrect aliasing for custom ops (#163227)
Ahead-Of-Time Inductor (AOTI)
  • Fix a bug from load_constants (#161887)
  • Fix wrong propagation of fallback_ops_dict in gen_aoti_c_shim (#159904)
  • Fix unbacked symint and memory leak in Inductor memory planning (#159839)
  • Fix memory leak in AOTI when calling aoti_torch_as_strided (#162118)
  • Explicitly delete wait_tensor returned tensor (#159502)
  • Fix memory leak from all_reduce (#159818)
Composability
  • Make functionalization ViewMeta serializable with pickle (#163769)
Export
  • Fix bug in constants lifting pass (#157719)
  • Fix from_node provenance in unlift pass (#157943)
  • Fix NaN serialization (#155359)
  • Fix deserialization for unbacked symbol ranges (#158681)
  • Fix runtime assert handling in deserialization (#159060)
  • Fix for FQN handling in unflattener (#159418)
  • Fix nn_module_stack for assert_tensor_metadata nodes (#159625)
  • Fix usage for move_to_device_pass (#159992, #160528, #162301)
  • Avoid name overwrites for aliased exported module parameters (#160600)
  • Avoid inling dynamo.disables in unflattening (#161306)
  • Fix deserialization issue for storage offset (#162172)
  • Remove .contiguous() when saving weights to raw bytes to preserve original storage size of tensor (#163587)
Quantization
  • Avoid NaN in fp8 output of CPU qlinear and qconv ops (#160957)
  • Fix segmentation fault when choose_qparams_optimized (#161966)
Foreach
  • chunk_size should always be int64_t for Foreach functors (#156872)
ONNX
  • Make onnx export SDPA match ATen behavior (#159973)
  • Fix rotary_embedding_23 implementation (#162865)
  • Fix export behavior when model has None as output (#160200)
  • Fix lower opset version support in dynamo=True (#161056)
  • Fix index_put_ usage (#161263)
C++ Extensions
  • Fix CPP extension distributed warning for TORCH_CUDA_ARCH_LIST to only log when running on non-distributed or on rank 0 (#162764)
C++ Frontend
  • Fix torch.utils.cpp_extension parser for clang version 20.1.7+libcxx (#157666)
  • Fix MakeTensor::computeStorageSize() calculation (#158690)
  • Fix static initialization order issue with AllocatorConfig (#159629)
Build Frontend
  • Turn on BUILD_BUNDLEPTXAS=1 to allow compile on newer GPUs(#163988)
CUDA
  • Handle uninitialized torch.backends.cuda.matmul.fp32_precision (#161102)
  • Fix nansum in non-JIT build (#158633)
  • Decrease launch bounds of CTCLoss backward for blackwell to avoid crash (#159522)
  • Implement workaround for cudaErrorNotSupported (#162412)
  • Fix missing __syncthreads in MultiMarginLoss backward (#158994)
  • Roll-back cuDNN frontend upgrade and update Meta registration due to compile issues (#163104)
  • Disable cuDNN for 3D convolutions with kernel size != 1 for cuDNN 9.8+ (#163581)
CPU
  • Add check so non-aarch64 platforms can hit MKLDNN path (#162168)
MPS
  • Fix batch norm incorrect gradient (#156867)
  • Do not crash if tensor dim > INT_MAX (#158824)
  • Avoid outputing zeros from exponential_ for MPS (#159386)
  • Fix MPS autocast for ConvTranspose3d (#160345)
  • Fix MPS conv3d autocast bias dtype mismatch (#160423)
  • Fix error check for torch.var on scalar (#160889)
  • Fix index_add for complex + int64, int64 input + zerodim index (#160926, #161511)
  • Fix constant_pad_nd_mps bug when pad is empty (#161149)
  • Fix index_select for scalar_types (#161206)
  • Fix index_copy for scalars and index_copy for strided indices (#161267, #161333)
  • Ensure that tensors are contiguous before using MPS linear kernel (#161641)
  • Address NaNs if SDPA is called with all values masked from query (#157727)
  • Fix invalid formatting (#158436)
  • Fix empty input in posneg functions (#161824)
  • Migrate round unary op to Metal (#161712)
  • Type-promote tensor-iterator common dtype (#160334)
  • Fix regression in 2.8.0 for scaled_dot_product_attention using MPS (#163598)
  • Chunk fillBuffer into 4Gb slices to avoid regression on MacOS 26 (#164108)
  • Fix latent bug that can result in segfault in CPP extensions (#164093)
ROCm
  • Fix Inductor with cudagraph trees hip:0 device error (#161221)
  • Fix some build failures and support some BLAS calls on Windows (#161981)
  • Fix undefined symbol linker error after exposing MIOpen symbols on Windows (#156479)
  • Fix finding ROCm/HIP version on Windows (#156486)
  • Fix LoadHIP handling of environment variable paths on Windows (#159080)
  • Add hipcc compatibility flags to cpp_extension.py on Windows (#159790)
  • In SDPA via AOTriton, logsumexp needs scaling back to natural base (#156903)
  • Check stream graph capture status in memcpy_and_sync inline function (#158165)
XPU
  • Fix cpp_extension compatibility with intel-deep-learning-essentials-2025.2 (#161012)
JIT
  • Make ErrorReport::CallStack thread-safe (#160386)
  • Fix RemoveProfileNodesAndSpecializeTypes handling for Tensor? that is resolved to None (#161538)

Performance

Optimizer
  • Use addmm to improve Newton–Schulz orthogonalization in Muon (#161379)
  • Avoid stream sync in SWA AveragedModel.update_parameters() (#157705)
Autograd
  • Fix SVD forward-mode AD multiplication priority (#161027)
Dynamo
  • Recursive dict tag optimization for faster guard evaluation (#159183)
Inductor
  • Improve performance of A16W4 and A16W8 GEMM template (#159127, #161148)
  • More aggressive persistent reduction (#161055)
  • Add a few outer dimension reduction cases for LOAF (#162028)
  • Fuse two RoPE kernels into a single kernel and improving runtime efficiency (#161420)
Export
  • Caching optimizations for placeholder naming pass (#158594)
  • Add Static Dispatch Kernel for fmod.Scalar and scale_gradient (#160654, #160454)
CUDA
  • Use a nonblocking copy to avoid stream synchronization for GPU tensor indexing with CPU mask (#156384)
  • Disable cudagraph GCs by default to improve capture performance (#158649)
Release Engineering
  • Upgrade to ROCm 6.4.1 and 6.4.2 patch releases (#156636, #158887, #158886, #158651, #159001)
  • Migrate RPyTorch ROCm CI to MI325 capacity (#159059, #159649, #161184)
  • Enable B200 PyTorch benchmark testing (#158011, #157341)
MPS
  • Optimize cummin/cummax metal kernels (#156794)
  • Speedup torch.full for 1-byte types (#158874)
  • Speedup argmax/argmin (#159524)
  • Improve performance of max_pool3d (#157875)
  • Avoid calling tensor ops in max_pool3d impl (#157874)
  • Move max_pool2d to Metal for stride != 1 (#157876)
ROCm
  • SDPA now uses AOTriton to 0.11b (#161754)
  • hipblaslt is used by default on gfx908 for ROCm >= 6.3 (#159092)
  • Enable miopen channels last 3d for conv and batchnorm (#160529)
  • Remove extra transposes in NHWC convolutions on MIOpen (#160435)
  • Remove extra sync in tensor.item() (#158486)
  • Elementwise and reduction kernel perf improvements (#159430, #159652, #160444, #160466, #161054, #161180, #161181)
  • Enable build of fbgemm_gpu genai sources for grouped GEMM support (#160676)
XPU
  • Enable tensor memory descriptor Triton template for Intel GPU (#161600)

Documentation

Python Frontend
  • Improve documentation for torch.lobpcg, torch.clone, torch.matmul, torch.max, torch.gather, torch.Tensor.scatter_, torch.empty_like, torch.randint, torch.mul, torch.min, torch.max. torch.sort, torch.full_like, torch.histogramdd, torch.hamming_window (#156139, #157007, #161424, #156153, #157929, #157920, #158050, #158731, #160312, #161539, #162051, #158275, #152682)
  • Remove torchscript related sections in serialization docs (#156648)
  • Fix typo in torch.set_float32_matmul_precision docs (#158191)
  • Fix docstring for torch.nn.utils.clip_grads_with_norm_ to reflect clamping behavior (#158200)
  • Fix the Doc issue on the description of edge_order in torch.gradient (#159130)
  • Add torch.segment_reduce docs (#154352)
  • Add examples to torch.is_floating_point and torch.is_complex docs (#161951)
torch.nn
  • Improve description of padding for avg_poolnd (#159142)
  • Improve CrossEntropyLoss docs with example of incorrect target specification (#155649)
  • Remove redundant dtype conversion in scaled_dot_product_attention example (#161613)
Optimizer
  • Document specific optimizer modules APIs e.g., torch.optim.adam.Adam, properly (#158483, #158669, #160194)
  • Add note for clarity in Adafactor doc #154862 (#155248)
  • Minorly improve zero_grad description (#161239)
Autograd
  • Improve torch.inference_mode docs and error message (#161164)
Distributed
c10d
  • Documented barrier collective's interaction with device_id (#159389)
  • Fix comment to match logic in distributed_c10d.py (#162158)
DTensor
  • Rewrote doc of TupleStrategy (#158132)
  • Documented redistribute_costs (#158495)
FullyShardedDataParallel (FSDP)
  • Removed FSDP1 developer note (#158991)
Profiler
  • Update PT2 Profiler Torch-Compiled Region Image (#158066)
  • Fix Experimental Config Documentatation(#156586)
  • Update README (#159816)
FX
  • Fix typos in torch/ (torch/fx/, #156604)
  • Add typing (#158450)
  • Fix typo in FX interpreter class docs (#162055)
  • Remove allow-untyped-defs from torch/fx/experimental/migrate_gradual_types/util.py (#157236)
Inductor
  • Add documentation for CUDAGraph partition (#159450)
Export
  • Update docs around draft export, dynamism, and PT2 Archive (#157750)
ONNX
  • Update export docstring (#162622)
  • Delete deprecated tutorial page link (#157310)
  • Filter out torchscript sentences (#158850)
  • Fix doc typo for symbolic_multi_out (#160702)
  • onnx.md to simplify deprecated entities (#159312)
  • Update export docstring and set fallback=False by default (#162622, #162726)
  • Fix typo in error message: summit -> submit (#162587)
Release Engineering
  • Add decorator to create deprecation warnings (#155127)
  • Add runnable code examples to export documentation (#158506)
  • Add developer notes for integrating new backends into PyTorch (#158644)
XPU
  • Update supported OS to Windows 11 & Ubuntu 24.04/25.04 for Intel client GPU (#161699)

Security

Python Frontend
  • Don't store flamegraph to tmp folder (#157374)

Developers

Python Frontend
  • Better sample inputs for addmm OpInfo (#160234)
Distributed
c10d
  • Add waitcounter for watchdog and heartbeat monitoring thread (#157480)
  • Made torch.distributed.breakpoint set a long timeout (#158481)
  • Add check_rng_sync util (#160283)
  • Add FlightRecorder support for ProcessGroupXCCL (#158568)
  • Add early_stop kwarg to torch.utils.checkpoint (#160781)
DTensor
  • Wrap sharding prop error with contextual exception (#161574)
  • Add check if tracing for sharding propagation to handle un-hashable keys in DTensor (#160798)
Device Mesh
  • Add error when users try to slice non contiguous flattened dim submesh (#157523)
  • Make the repr shorter when debug ENV not set (#158822)
ShardedTensor
  • Make error message descriptive in ShardedTensor creation (#150627, #159423)
Pipeline Parallelism (PP)
  • Add profiling to schedule execution (#160753)
FX
  • Consolidate stack trace in Tracer (#156257, #157302, #158266)
  • Separate provenance tracking to different levels (#160383, #158399, #158796, #159484)
  • Fix register_foward_pre_hook not supported on ScriptModule error (#156904)
  • Add __eq__ function to NodeSource (#158170)
  • Add __hash__ function to NodeSource (#158322)
  • Cache dict and string rep for better perf in NodeSource (#158372)
  • Recover node source from dict (#158373, #158473)
  • Include error stacktrace and graph module in tlparse error (#158469)
  • Add expanded_def option for FX printing, render descriptor, update tests (#158708)
  • Remove co_lnotab in favor of co_linetable (#159227)
  • Remove duplicate imports (#161685)
  • Include Output tensor metadata for CompiledFxGraph (#159311)
Inductor
  • Deprecate allow_tf32 in tl.dot(..., allow_tf32=...), use tl.dot(..., input_precision=...) (#160711)
  • Log autotune choices and benchmark result to scuba/chrome trace (#159496)
  • Add TLParse artifact for logging runtime of collective and compute ops (#159730)
  • Call jit_post_compile_hook within Inductor Triton Kernel compile path (#161443)
  • Prune configs that require more shared memory than the hardware limit (#161996)
  • Runtime estimations using nccl estimator on mm only benchmark mode (#161405)
  • Don't use torch.backends.cuda.matmul.allow_tf32 in Inductor cache key (#159480)
Ahead-Of-Time Inductor (AOTI)
  • Better error message when no .so/cpp files are found (#156863)
  • Clean up old APIs in AOTI c shim (#158400)
  • Add Inductor provenance mapping for cpp extern kernel (#161656, #162069)
  • Print out error msg when nvcc compiler fails (#157203)
  • Add kernel information JSON generation for AOTI packages (#160540)
Composability
  • Stop suggesting to use guard_size_oblivious on data dependent errors (#160510)
  • Avoid unnecessary slices resulting in data-dependent errors (#157528)
Quantization
  • Revamp dtype documentation (#156087)
  • Use new type statement to fix public API of types (#158487)
Dataloader Frontend
  • Add torch.utils.data samplers benchmark script (#156974)
  • Add torch.utils.data.Dataloader benchmark script (#159432)
Release Engineering
  • Replace setup.py develop with pip install -e for development builds (#155998, #156027, #156710) (#156709)
XPU
  • Upgrade Intel GPU software stack package to intel-deep-learning-essentials-2025.2 (#158733)
View originalPermalink
v2.8.0

PyTorch 2.8.0 Release

PyTorch 2.8.0 Release Notes

Highlights

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Tracked Regressions

Windows wheel builds with CUDA 12.9.1 stack overflow during build (#156181)

Due to a bug introduced in CUDA 12.9.1, we are unable to complete full Windows wheel builds with this version, as compilation of torch.segment_reduce() crashes the build. Thus, we provide a wheel without torch.segment_reduce() included in order to sidestep the issue. If you need support for torch.segment_reduce(), please utilize a different version.

Backwards Incompatible Changes

CUDA Support
Removed support for Maxwell and Pascal architectures with CUDA 12.8 and 12.9 builds (#157517, #158478, #158744)

Due to binary size limitations, support for sm50 - sm60 architectures with CUDA 12.8 and 12.9 has been dropped for the 2.8.0 release. If you need support for these architectures, please utilize CUDA 12.6 instead.

Python Frontend
Calling an op with an input dtype that is unsupported now raises NotImplementedError instead of RuntimeError (#155470)

Please update exception handling logic to reflect this.

In 2.7.0

try:
    torch.nn.Hardshrink()(torch.randint(0, 5, (10,)))
except RuntimeError:
    ...

In 2.8.0

try:
    torch.nn.Hardshrink()(torch.randint(0, 5, (10,)))
except NotImplementedError:
    ...
Added missing in-place on view check to custom autograd.Function (#153094)

In 2.8.0, if a custom autograd.Function mutates a view of a leaf requiring grad, it now properly raises an error. Previously, it would silently leak memory.

   class Func(torch.autograd.Function):
        @staticmethod
        def forward(ctx, inp):
            inp.add_(1)
            ctx.mark_dirty(inp)
            return inp

        @staticmethod
        def backward(ctx, gO):
            pass

    a = torch.tensor([1.0, 2.0], requires_grad=True)
    b = a.view_as(a)
    Func.apply(b)

Output:

Version 2.7.0

Runs without error, but leaks memory

Version 2.8.0

RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation
An error is now properly thrown for the out variant of tensordot when called with a requires_grad=True tensor (#150270)

Please avoid passing an out tensor with requires_grad=True as gradients cannot be computed for this tensor.

In 2.7.0

a = torch.empty((4, 2), requires_grad=True)
b = torch.empty((2, 4), requires_grad=True)
c = torch.empty((2, 2), requires_grad=True)
# does not error, but gradients for c cannot be computed
torch.tensordot(a, b, dims=([1], [0]), out=c)

In 2.8.0

a = torch.empty((4, 2), requires_grad=True)
b = torch.empty((2, 4), requires_grad=True)
c = torch.empty((2, 2), requires_grad=True)
torch.tensordot(a, b, dims=([1], [0]), out=c)
# RuntimeError: tensordot(): the 'out' tensor was specified and requires gradients, and
# its shape does not match the expected result. Either remove the 'out' argument, ensure
# it does not require gradients, or make sure its shape matches the expected output.
torch.compile
Specialization of a tensor shape with mark_dynamic applied now correctly errors (#152661)

Prior to 2.8, it was possible for a guard on a symbolic shape to be incorrectly omitted if the symbolic shape evaluation was previously tested with guards suppressed (this often happens within the compiler itself). This has been fixed in 2.8 and usually will just silently "do the right thing" and add the correct guard. However, if the new guard causes a tensor marked with mark_dynamic to become specialized, this can result in an error. One workaround is to use maybe_mark_dynamic instead of mark_dynamic.

See the discussion in issue #157921 for more context.

Version 2.7.0

import torch

embed = torch.randn(2, 8192)
x = torch.zeros(8192)

torch._dynamo.mark_dynamic(x, 0)

@torch.compile
def f(embedding_indices, x):
    added_tokens_mask = torch.where(x > 10000, 1, 0)
    ei = torch.narrow(embedding_indices, 1, 0, x.size(0))
    return ei.clone()

f(embed, x)

Version 2.8.0

import torch

embed = torch.randn(2, 8192)
x = torch.zeros(8192)

torch._dynamo.maybe_mark_dynamic(x, 0)

@torch.compile
def f(embedding_indices, x):
    added_tokens_mask = torch.where(x > 10000, 1, 0)
    ei = torch.narrow(embedding_indices, 1, 0, x.size(0))
    return ei.clone()

f(embed, x)
Several config variables related to torch.compile have been renamed or removed
  • Dynamo config variable enable_cpp_framelocals_guard_eval has changed to no longer have any effect (#151008).
  • Inductor config variable rocm.n_max_profiling_configs is deprecated (#152341). Instead, use ck-tile based configs rocm.ck_max_profiling_configs and rocm.ck_tile_max_profiling_configs.
  • Inductor config variable autotune_fallback_to_aten is deprecated (#154331). Inductor will no longer silently fall back to ATen. Please add "ATEN" to max_autotune_gemm_backends for the old behavior.
  • Inductor config variables use_mixed_mm and mixed_mm_choice are deprecated (#152071). Inductor now supports prologue fusion, so there is no need for special cases now.
  • Inductor config setting descriptive_names = False is deprecated (#151481). Please use one of the other available options: "torch", "original_aten", or "inductor_node".
  • custom_op_default_layout_constraint has moved from inductor config to functorch config (#148104). Please reference it via torch._functorch.config.custom_op_default_layout_constraint instead of torch._inductor.config.custom_op_default_layout_constraint.
  • AOTI config variable emit_current_arch_binary is deprecated (#155768).
  • AOTI config variable aot_inductor.embed_cubin has been renamed to aot_inductor.embed_kernel_binary (#154412).
  • AOTI config variable aot_inductor.compile_wrapper_with_O0 has been renamed to compile_wrapper_opt_level (#148714).
Added a stricter aliasing/mutation check for HigherOrderOperators (e.g. cond), which will explicitly error out if alias/mutation among inputs and outputs is unsupported (#148953, #146658).

For affected HigherOrderOperators, add .clone() to aliased outputs to address this.

Version 2.7.0

import torch

@torch.compile(backend="eager")
def fn(x):
    return torch.cond(x.sum() > 0, lambda x: x, lambda x: x + 1, [x])

fn(torch.ones(3))

Version 2.8.0

import torch

@torch.compile(backend="eager")
def fn(x):
    return torch.cond(x.sum() > 0, lambda x: x.clone(), lambda x: x + 1, [x])

fn(torch.ones(3))
guard_or_x and definitely_x have been consolidated (#152463)

We removed definitely_true / definitely_false and associated APIs, replacing them with guard_or_true / guard_or_false, which offer similar functionality and can be used to achieve the same effect. Please migrate to the latter.

Version 2.7.0

from torch.fx.experimental.symbolic_shapes import definitely_false, definitely_true

...
if definitely_true(x):
  ...

if definitely_false(y):
  ...

Version 2.8.0

from torch.fx.experimental.symbolic_shapes import guard_or_false, guard_or_true

...
if guard_or_false(x):
  ...

# alternatively: if guard_or_false(torch.sym_not(y))
if not guard_or_true(y):
  ...
torch.export
torch.export.export_for_inference has been removed in favor of torch.export.export_for_training().run_decompositions() (#149078)

Version 2.7.0

import torch

...
exported_program = torch.export.export_for_inference(mod, args, kwargs)

Version 2.8.0

import torch

...
exported_program = torch.export.export_for_training(
    mod, args, kwargs
).run_decompositions(decomp_table=decomp_table)
Switched default to strict=False in torch.export.export and export_for_training (#148790, #150941)

This differs from the previous release default of strict=True. To revert to the old default behavior, please explicitly pass strict=True.

Version 2.7.0

import torch

# default behavior is strict=True
torch.export.export(...)
torch.export.export_for_training(...)

Version 2.8.0

import torch

# strict=True must be explicitly passed to get the old behavior
torch.export.export(..., strict=True)
torch.export.export_for_training(..., strict=True)
ONNX
Default opset in torch.onnx.export is now 18 (#156023)

When dynamo=False, the default ONNX opset version has been updated from 17 to 18. Users can set opset_version to explicitly select an opset version.

Version 2.7

# opset_version=17
torch.onnx.export(...)

Version 2.8

# To preserve the original behavior
torch.onnx.export(..., opset_version=17)

# New: opset_version=18
torch.onnx.export(...)
The JitTraceConvertStrategy has been removed (#152556)

Support for JIT traced and scripted modules in the ONNX exporter when dynamo=True has been removed. You are encouraged to export an nn.Module directly, or create an ExportedProgram using torch.export before exporting to ONNX.

onnxscript>=0.3.1 is required for the dynamo=True option (#157017)

You must upgrade onnxscript to version 0.3.1 or higher for it to be compatible with PyTorch 2.8.

Build Frontend
Removed the torch/types.h include from Dispatcher.h (#149557)

This can cause build errors in C++ code that implicitly relies on this include (e.g. very old versions of torchvision).

Note that Dispatcher.h does not belong as an include from torch/types.h and was only present as a short-term hack to appease torchvision. If you run into torchvision build errors, please update to a more recent version of torchvision to resolve this.

Upgraded DLPack to 1.0 (#145000)

As part of the upgrade, some of the DLDeviceType enum values have been renamed. Please switch to the new names.

Version 2.7.0

from torch.utils.dlpack import DLDeviceType

d1 = DLDeviceType.kDLGPU
d2 = DLDeviceType.kDLCPUPinned
...

Version 2.8.0

from torch.utils.dlpack import DLDeviceType

d1 = DLDeviceType.kDLCUDA  # formerly kDLGPU
d2 = DLDeviceType.kDLCUDAHost  # formerly kDLCPUPinned
...
NVTX3 code has been moved from cmake/public/cuda.cmake to cmake/Dependencies.cmake (#151583)

This is a BC-breaking change for the build system interface. Downstream projects that previously got NVTX3 through cmake/public/cuda.cmake (i.e.. calling find_package(TORCH REQUIRED)) will now need to explicitly configure NVTX3 support in the library itself (i.e. use USE_SYSTEM_NVTX=1). The change is to fix the broken behavior where downstream projects couldn't find NVTX3 anyway due to the PROJECT_SOURCE_DIR mismatch.

Version 2.7.0:

  • A downstream project using -DUSE_SYSTEM_NVTX would be able to find NVTX3 and torch::nvtx3 via PyTorch's cmake/public/cuda.cmake logic.
  • A downstream project NOT using -DUSE_SYSTEM_NVTX would encounter build errors with CUDA 12.8 or above.

Version 2.8.0:

  • A downstream project using -DUSE_SYSTEM_NVTX will not be able to find NVTX3 or torch::nvtx3 via PyTorch's cmake/public/cuda.cmake. The downstream project now needs to explicitly find NVTX3 and torch::nvtx3 by implementing the same logic in PyTorch's cmake/Dependences.cmake.
  • A downstream project NOT using -DUSE_SYSTEM_NVTX will proceed building without NVTX unless another part of the build process re-enables NVTX.

Deprecations

MPS support for MacOS Ventura will be removed in 2.9

PyTorch 2.8 is the last release that will support GPU acceleration on MacOS Ventura. In the next release (2.9), MacOS Sonoma (released in Sept. 2023) or above will be required to use the MPS backend.

torch.ao.quantization is deprecated and will be removed in 2.10 (#153892)

To migrate:

  • Eager mode quantization (torch.ao.quantization.quantize, torch.ao.quantization.quantize_dynamic)
    • Weight-only and dynamic quantization: use torchao eager mode quantize_.
    • Static quantization: use torchao PT2E quantization.
  • FX graph mode quantization (torch.ao.quantization.quantize_fx.prepare_fx, torch.ao.quantization.quantize_fx.convert_fx): use torchao PT2E quantization (torchao.quantization.quantize_pt2e.prepare_pt2e, torchao.quantization.quantize_pt2e.convert_pt2e).

Note that PT2E quantization has been migrated to torchao (https://github.com/pytorch/ao/tree/main/torchao/quantization/pt2e). See https://github.com/pytorch/ao/issues/2259 and https://docs.pytorch.org/ao/main/quick_start.html#pytorch-2-export-quantization for more details.

The dynamo=False (current default) option for torch.onnx.export is deprecated (#152478, #155580)

The default will be dynamo=True starting from PyTorch 2.9. You are encouraged to migrate to use the dynamo=True option in torch.onnx.export. This flag makes torch.export.export the default export path, replacing TorchScript.

To maintain the old behavior, set dynamo=False explicitly. You are encouraged to also experiment with the fallback=True option that will make the exporter fall back to the dynamo=False path if there are errors.

New Features

CUDA
  • Support capture of event record and wait in CUDAGraphs for timing (#155372)
torch.compile
Dynamo
  • Added support for hierarchical compilation via nested_compile_region (#156449)
  • Allow guards to be dropped with custom filter functions via guard_filter_fn (#150936)
  • Added dont_skip_tracing decorator to skip over most Dynamo skipfiles rules (#150586)
Inductor
  • Added support for mapping a Dynamo graph to multiple different Inductor graphs, which can be optimized separately (#147648, #147038)
torch.export
  • Introduced draft-export, an export variant designed to consistently produce a graph and generate a debugging report of issues encountered during tracing (#152637, #153219, #149465, #153627, #154190, #155744, #150876, #150948, #151051, #151065, #150809, #151797)
Ahead-Of-Time Inductor (AOTI)
  • Added support for TorchBind objects (#150196, #154265)
  • Added config variable aot_inductor.model_name_for_generated_files for specifying model name (#154129)
MPS
  • MPSInductor: torch.compile for Apple GPUs (#150121, #149342, #151449, #151754, #149687, #149180, #149221, #153598, #152788, #153787, #152214, #151152, #155891, #154578, #151272, #151288, #153997, #151871, #153362, #156566, #150661, #153582)
ONNX

  • Added new strategy draft_export (#147529, docs) to provide debugging information upon data-dependent / constraint errors when obtaining an ExportedProgram with torch.onnx.export

  • Added support for symbolic operators in the dynamo=True export path (#148905, #149678, #150038, docs). Two operators torch.onnx.ops.symbolic and torch.onnx.ops.symbolic_multi_out are defined to allow you to create symbolic ONNX operators directly in your PyTorch models. You can use them in a forward method:

def forward(self, x: torch.Tensor) -> torch.Tensor:
    # Optionally use is_in_onnx_export to control the behavior during onnx export

    if torch.onnx.is_in_onnx_export():
        # Create a symbolic ONNX operator with the name "CustomOp" in the "custom_domain" domain.
        # The output tensor will have the specified dtype and shape
        return torch.onnx.ops.symbolic(
            "custom_domain::CustomOp",
            (x,),
            dict(attr_key="attr_value"),
            dtype=x.dtype,
            shape=x.shape,
            version=1,
        )
    else:
        return x
Python Frontend
  • Added Generalized Pareto Distribution (GPD) (#135968)
Quantization
  • Introduced torch.float4_e2m1fn_x2 dtype (#148791)
XPU
  • Support Intel distributed backend (XCCL) (#141856)
  • Support SYCL kernels through C++ extension (#132945)

Improvements

Build Frontend
  • Removed outdated warning about TORCH_CUDA_ARCH_LIST (#152715, #155314)
  • Made Eigen an optional build dependency (#155955)
  • Updated CUTLASS to 3.9.2 (#152779)
Composability
  • Enhanced custom op support with serializable op profiles and fake registration overrides (#151817, #150807, #150806)
C++ Frontend
  • Exposed bicubic mode for torch::nn::functional::grid_sample (#150817)
CUDA
  • Introduced no_implicit_headers mode for load_inline() on custom CUDA extensions (#149480)
  • Support large batch sizes in SDPA memory-efficient attention backend (#154029, #154663)
  • Fixed invalid indexing in SDPA memory-efficient attention backward (#155397)
  • Support SDPA attention backends on sm121 (DGX Spark) (#152314)
  • Added FP8 row-wise scaled-mm for sm12x (GeForce Blackwell) (#155991)
cuDNN
  • Updated cuDNN frontend version to 1.12 (#153888)
Distributed
c10d
  • Enhanced TCPStore with clone and queuing features (#150966, #151045, #150969, #151485)
  • Added a collective time estimator for NCCL comms (#149343)
  • Made getDefaultBackend more fault tolerant without relying on exceptions (#149152)
  • Specified the default PyTorch Distributed backend for MPS (#149538)
  • Supported masterListenFd in TCPStoreLibUvBackend (#150215)
  • Used shared stores in gloo (#150230)
  • Improved FR dump robustness with all watchdog broadcast wait, reduce dump timeout and shrinked mutex range (#150652, #151329, #155949)
  • Added the record of each individual collective being coalesced in FR (#151238)
  • Implemented safer book-keeping of NCCL communicators (#150681)
  • Clarified behavior of TORCH_NCCL_USE_TENSOR_REGISTER_ALLOCATOR_HOOK (#150682)
  • Registered also future allocations in mempool with NCCL (#150684)
  • Avoided computing global_rank when group_rank is used (#151373)
  • Exposed NCCL communicator from ProcessGroupNCCL via an unsafe API (#152496)
  • Added split sizes info dump for uneven all2all bw calculation (#151438)
  • Made FR vendor neutral so that other backends can use it and integrated into gloo. (#152585, #152563, #154929, #152614)
  • Added needs_contiguous_strides tag in functional collective (#153399, #153523)
  • Allowed split_group to work with non-nccl backends (#152175)
  • Simplified new_subgroups() by using new_subgroups_by_enumeration() (#153843)
  • Made only current thread allocate to pool in ProcessGroupNCCL (#153990)
  • Enabled using c10::Half for gloo (#153862)
  • Released GIL in PG destructor (#154976)
  • Enhanced get_process_group_ranks() to accept group=None (#154902)
  • Skipped updating the default device distributed backend if already registered (#155320)
  • Enabled querying the build and runtime NCCL versions (#156305)
  • Disabled NCCL NVLS when using deterministic mode (#156381)
  • Made init_process_group support index-only device id (#156214)
  • Support enabling / disabling NaN detector per-ProcessGroup (#151723)
  • Added support for reduce_scatter and ReduceOp::AVG in ProcessGroupGloo (#149781, #149869)
  • Added FP8 support in ProcessGroupNCCL (#152706)
  • Added ibverbs backend in gloo and enabled gloo CUDA when used with a backend that supports GPUDirect (#153015, #153425, #153406)
DeviceMesh
  • Improved device selection logic (#150897)
DistributedDataParallel (DDP)
  • Added one option to allow skipping all reduce unused parameters (#151503)
  • Added check on received data to avoid segfault in the DDP reducer (#152143)
  • Propagated use_python_reducer to C++ reducer (#152735) DistributedStateDict (DSD)
  • Supported non-tensor-data write_size in planner write items (#149699)
  • Skip popping meta device tensors (#153185)
DTensor
  • Made StridedShard support uneven sharding (#150490)
  • Added op support for torch.cumsum (#151071)
  • Added DTensor redistribute fwd/bwd datatype conversion to enable SimpleFSDP mixed precision training (#150740)
  • Added rich support to torch.distributed.tensor.debug.visualize_sharding (#152027)
FullyShardedDataParallel2 (FSDP2)
  • Added PrivateUse1 backend in FSDP collectives and device type to pre forward hook (#147260, #149487)
  • Added set_reshard_after_forward (#149103)
  • Allowed different dtypes for no grad model params (#154103)
  • Respected reshard_after_forward=True for root model and kept root unsharded when not specifying reshard_after_forward (#154704, #155319)
  • Allowed forcing FSDP2 to always use SUM reductions (#155915)
  • Made assert on all_reduce_event only if it's not CPU device (#150316)
  • Enabled NCCL zero-copy (user buffer registration) for FSDP2 (#150564)
Pipeline Parallelism
  • Added schedule visualizer (#150347)
  • Allowed unused kwargs in ZB path (#153498)
  • Added get_pipeline_order() for Gpipe and 1F1B (#155935)
ShardedTensor
  • Added support for 0-size ShardedTensor and recalculated metadata from all_gather (#152583)
TensorParallel
  • Added a ParallelStyle PrepareModuleInputOutput (#150372)
torchelastic
  • No shutdown of rendezvous on leaving workers (#152525)
torch.compile
Dynamo
  • Improved tracing support for python sets, tensor subclasses with __torch_function__, and namedtuple subclasses (#153150, #149792, #153982)
  • Eliminated all Compiled Autograd dynamic shapes recompiles for compile time reduction (#151962, #152119, #151962, #149707, #149709, #148799, #148801)
  • Added reason field to torch.compiler.disable (#150341)
  • Removed lru_cache warnings for functions in the top-level torch namespace (#157718)
Inductor
  • Added block sparse support for FlexAttention on CPU (#147196)
  • Introduced new config settings:
    • aot_inductor.custom_ops_to_c_shims and aot_inductor.custom_op_libs: allow for specifying custom op C shim (#153968)
    • max_fusion_buffer_group_pairwise_attempts: limits fusions to specified node distance (#154688)
    • cuda.cutlass_enabled_ops: controls CUTLASS operation selection (#155770)
    • triton.cudagraph_capture_sizes: allows specifying certain shapes for which to capture CUDAGraphs; skips CUDAGraphs for other shapes (#156551)
    • use_static_cuda_launcher: enables launching compiled triton statically to improve cold start times (#148890)
    • assume_unaligned_fallback_output: allows inductor to track unaligned outputs (#150777)
    • cuda.cutlass_tma_only: controls whether or not to only use TMA-compatible kernels in CUTLASS (#152815)
    • static_launch_user_defined_triton_kernels: enables statically launching user defined triton kernels (#153725)
    • precompilation_timeout_seconds: controls the timeout on precompilation (#153788)
    • disable_decompose_k: disables new DecomposeK GEMM Kernels (#154421)
    • min_num_split: sets the minimum number of splits in a split reduction (#155941)
    • max_autotune_flex_search_space: allows specifying the size of the search space for flex attention autotuning (#156307)
  • Introduced environment variable LOG_AUTOTUNE_RESULTS for autotune log (#156254)
  • Improved numerical stability of CPU Welford reduction for normalizations (#145061)
torch.export
  • Improved handling of builtin ops (min, max, math.pow) (#151348)
  • Added min/max ranges for dim hints (#149590)
  • Allow registering normal classes to pytree.register_dataclass (#147752)
  • Allow specifying integer inputs as dynamic (#151842)
  • Inline jit.scripted functions in export (#155180)
  • Pretty printing for graph signature (#149710)
Ahead-Of-Time Inductor (AOTI)
  • Support for device-side TMA (#157241)
  • Added num_runners to AOTIModelPackageLoader (#149364)
FX
  • Updated codegen compare op to == (#150611)
  • Map names to operand indices when const folding submodules (#150692)
  • Improved stacktrace when tracing (#151029, #155486)
  • Support edge dialect ops in normalize_function (#143689)
  • Fixed path naming in minifier (#153130)
  • Added graph_code_verbose_log artifact for FX passes (#153775)
  • Improved cache key graph printing performance (#151928)
  • Added flag to fx.passes.split_module to normalize input names (#157793)
Linear Algebra Frontend
  • Add tensor overlap check for cross (#154999)
MPS
  • Added support for a number of torch.special operations as well as index_copy, hardshrink, rsub, col2im, and isin (#149174, #149203 #149123, #149368, #149378, #149563, #149687, #149705, #149783, #149407/#149680, #150279, #151754, #153786, #154326, #155304, #156263, #155382, #154010, #149816, #152282, #156090, #150060, #151600, #155002, #154671)
  • Extended dtype support for:
    • index_put with half precision floats (#151869)
    • ConvTranspose3D with FP32 and complex (#154696)
    • log1p and sigmoid with int64 (#151791)
  • Compute activation kernels at float precision (#155735)
Nested Tensor (NJT)
  • Fixed contiguity in NJT string representation (#153529)
torch.nn
  • Added warning for module full backward hook when no input requires gradient (#155339)
  • Added Half support for weight_norm on CPU (#148878)
ONNX
  • Updated ONNX to 1.18 (#152200)
  • Added support for opsets (18-23) when dynamo=True (#149901, #154596)
  • Added float4 support (#151069, #156353)
  • Added support for ONNX operators Attention-23 and RotaryEmbedding-23 as native PyTorch ops (#156431, #156367, #154745)
  • Added support for torch.scan (#154513)
  • Added support for 0/1-sized example inputs on dynamic dimensions (#155717)
  • Add group_norm support from opset 21 (#152138)
  • Added asdict method to VerificationInfo class (#151024)
  • Support running bfloat16 models with ONNX Runtime (#149646)
  • Updated ONNX program doc formatting and improve robustness (#151623)
  • Updated dynamic_shapes behavior to use torch.export.dim.DYNAMIC (#153065)
  • Set the name of the producing node using the value name (#155413)
  • Improved support for symbolic operators sym_float, sym_not, sym_min, sym_max (#153200, #152111, #152196)
Optimizer
  • Added TensorLR variant for fused Adagrad on CPU (#153078)
  • Convert tensor lr to 0-dim as needed for the optimizer to normally work (#145674)
  • Added lr_lambda type check in MultiplicativeLR (#151973)
Profiler
  • Added support for on-demand memory snapshot (#150559)
  • Added PT2 compile context to visualizer (#152862)
  • Added PT2 to memory snapshot (#152707)
  • Added flag to toggle global and local callbacks for annotations (#154932)
  • Pass overload names to Kineto (#149333)
  • Set duration to -1 for unfinished CPU events (#150131)
  • Start at index with most events (#154571)
Python Frontend
  • Introduced torch.AcceleratorError (#152023)
  • Implemented Size.__radd__() (#152554)
  • Updated get_default_device() to also respect torch.device context manager (#148621)
Quantization
  • Improved x86 PT2E quantization support with new uint8 ops (pointwise mul / add / add_relu and batch_norm2d), qconv1d-relu fusion, and lowering pass (#151112, #152411, #152811, #150751, #149708)
  • Support boolean tensor for torch.fused_moving_avg_obs_fake_quant on CUDA (#153699)
Release Engineering
  • Updated gcc11 to gcc13 in manylinux images (#152825, #152825, #150635, #158445)
  • Updated to cmake 3.27.2 (#154783, #150549, #153380)
ROCm
  • Allow user to override default flags for cpp_extension (#152432)
  • Enabled support for sparse compressed mm/bmm/addmm (#153262)
Sparse Frontend
  • Enabled sparse compressed tensor invariant checks for PrivateUse1 extension (#149374)
torch.func
  • Add batching rules for ops: torch.Tensor.scatter_add_ (#150543), torch.matrix_exp (#155202)
XPU
  • Support safe softmax, GQA, fp32 causal mask for SDP and increase maximum head dim from 256 to 576 on Intel GPU (#151999, #150992, #152091)
  • Add memory reporting to Memory Profiler for Intel GPU (#152842)
  • Support Intel GPU profiler toggle functionality (#155135)
  • Support distributed memory tracker integration for Intel GPU (#150703)
  • Improved error handling and reporting in Intel GPU CMake files (#149353)
  • Support embed_cubin and multi_arch_kernel_binary options in AOTI for Intel GPU (#154514, #153924)
  • Added generic and Intel GPU specific Stream and Event in UserDefineClass (#155787)
  • Support int4 WOQ GEMM on Intel GPU (#137566)

Bug Fixes

Build Frontend
  • Support builds with CMake-4.x (#150203)
  • Fixed fbgemm build with gcc-12+ (#150847)
  • Force build to conform to C++ standard on Windows by adding /permissive- flag (#149035)
Composability
  • Fixed support for 1-element tuple returns from custom ops (#155447)
  • Avoid overflow in torch.norm for scalar input (#144073)
CPU (x86)
  • Fixed apparent copy-paste bug in log_softmax reduced-precision fp kernel (#156379)
CUDA
  • Fixed deterministic indexing with broadcast (#154296)
  • Fixed torch.backends.cuda.matmul.allow_fp16_accumulation crash when using cuBLASLt (#153083)
  • Enable AsyncMM on Blackwell (#153519)
  • Fixed torch.cuda.MemPool for multithreaded use-cases (#153356)
  • Fix to avoid calling sum() on a default-constructed gamma / beta in layer_norm (#156600)
  • Avoid hangs by erroring out for negative offsets or K=0 in grouped GEMMs (#153226)
  • Don't error out in empty_cache under mempool context (#158180)
Distributed
c10d
  • Fixed extra CUDA context created by barrier (#149144)
  • Fixed the logic to use group rank instead of global rank when possible (#149488)
  • Fixed ET trace collection of all_to_all (#149485)
  • Disabled start event recording for coalesced col and improved profile title (#150863)
  • Fixed connection reset in tcp store (#150987, #151052)
  • Fixed unused group input argument in new_subgroups() (#152765, #153798)
  • Fixed tcp init when using port 0 (#154156)
  • Adopted a vector to temporarily keep the reference to future object to avoid blocking inside Flight Recorder (#156653)
Distributed Checkpointing (DCP)
  • Fixed to use global coordinator rank in broadcast_object util function (#155912)
DistributedDataParallel (DDP)
  • Fixed DDPOptimizer issue on static tensor index (#155746)
DTensor
  • Fixed local_map with multi-threading (#149070)
  • Fixed new_local_tensor in redistribute be None case (#152303)
  • Fixed bug visualizing 1D Tensor using rich (#152871)
Pipeline Parallelism
  • Optimized memory usage by releasing output memory earlier (#153383)
RPC
  • Made torch importable if compiled without TensorPipe (#154382)
ShardedTensor
  • Fixed sharded tensor gather when a local tensor on certain ranks has zero elements (#150914)
TensorParallel
  • Turn async-TP applicability asserts back into silent skips (#158736)
torch.compile
Dynamo
  • Eliminated silent incorrectness issues in the Compiled Autograd initial trace (#149014, #155521, #155289, #149336)
  • Fixed various tracing errors involving einops, dict(mapping_proxy), and the FlexAttention HOP (#157754, #157515, #157519)
  • Fixed unpack hook semantics for memory savings in checkpointing and offloading for Compiled Autograd (#147242, #153300)
  • Fixed sources for dataclass defaults and the lru_cache method (#158689, #157308)
  • Fixed spammy errors when an invalid TORCH_LOGS argument is passed (#151678)
Inductor
  • Support special kwargs in AMD triton configs (#154605)
  • Fixed minifier when one has multiple Python runtimes (#155918)
  • Bug fix for int8 GEMM compensation epilogue (#152408)
torch.export
  • Fixed tracing of the following: aten.is_nonzero (#149637), torch.bincount() (#152497), aten.div (#150874) slicing (#150104), and attn_mask (#158618), aten.to (#153972), scalar tensor construction (#154661)
  • Fixed dynamic_shapes spec for kwargs (#148772, #149528, #150103)
  • Fixed input bugs in unflattener (#149206, #153474, #153000)
  • Fix nonstrict tracing of functools.partial (#153408), and higher order ops (#149295)
  • Fixed serialization/deserialization of None inputs (#150515), math module (#154643), call_torchbind (#155647), and enums (#154821)
  • Fixed state dict modification in run_decompositions (#151436)
  • Fixed subclass access custom op bug (#149698)
Ahead-Of-Time Inductor (AOTI)
  • Fixed AOTI update_constant_buffer issue (#149243)
  • Fixed a memory leak in model_package_loader (#152334)
  • Don't alloc weights in AOTIModel if they don't exist (#152692)
  • Fixed state of ConstantFolding (#153152)
  • Fixed index offset for optional tensor return (#155073)
  • Fixed float8 type printing for min/max value printing (#154466)
Linear Algebra Frontend
  • Fix to workaround LAPACK workspace size being returned as a floating point value (#149682)
  • Fixed the accumulation type for dot and gemv (#152676)
  • Fixed torch.lobpcg to compute same largest eigenvalue as scipy and np.linalg.eig (#152789)
  • Fixed 32-bit indexing overflows in ReducedPrecisionGemV (#150949)
MPS
  • Fixed various op support issues: unary/binary ops with 2**32+ element inputs, binary ops with inputs with different dtypes, ops with complex scalar inputs, cholesky decomp, floor_divide type promotion, index_kernel with large inputs, lerp with complex inputs, logit with half/bfloat16 inputs, SDPA memory leak, torch.special.entr, tri[ul], matrix inversion with N>1024, and where with non-contiguous cond (#152479, #155183, #149233, #151176, #151282, #158239, #152371, #149974, #158237, #146754, #158867, #155184, #152204)
torch.nn
  • Fixed load_state_dict behavior for nn.LazyLinear (#147599)
ONNX
  • Fixed bfloat16 support in onnx_program callable (#151121)
  • Produce correct dtypes for bf16/f8 in IR TorchTensor (#151259)
  • Preserve all legacy exporter params in fallback (#156659)
  • Fixed 4D tensor conversion for SDPA (#157509)
Optimizer
  • Fixed bug where lr_scheduler unexpectedly calls step() when init argument last_epoch > -1 (#149312)
  • Fixed CosineAnnealingWarmRestarts resetting T_cur (#151289)
Profiler
  • Fixed empty C call queue in python tracer (#150370)
  • Removed decref from python context in python tracer (#151625)
  • Enable all configured activities in CUPTI Range Profiler mode (#154749)
Python Frontend
  • Fixed segfault during numpy string tensor conversion (#155364)
  • Added checks for empty tensor list (#155383)
  • Fixed sample validation for MixtureSameFamily distribution (#151317)
  • Fixed bug where creating a second Wishart or Uniform distribution modifies constraints on the first (#154361)
  • Fix to properly export torch::utils::tensor_to_numpy symbol (#154178)
  • Fixed torch.[con]cat[enate] to avoid crashing on empty inputs (#155460)
  • Unify torch.tensor and torch.ops.aten.scalar_tensor behavior (#158655)
Release Engineering
  • Checkout optional submodules when publishing a release tarball (#156615)
  • Fixed MacOS MP hang in Python-3.12+ (#155698)
  • Fixed static functions when using module in MSVC (#148675)
  • Fixed VS2022-caused AVX512 illegal instruction issue (#153480)
ROCm
  • Fixed build error for opportunistic fastatomics with newer compilers (#152841)
TunableOp
  • More TF32 support (#149088)
  • Fixed offline tuning for ScaledGEMM (#149677)
  • Fixed row-wise ScaledGEMM (#152403)
  • Support submatrices in offline tuning for ROCm (#151138)
Vulkan
  • Fixed torch.is_vulkan_available() on Mac (#155595)
XPU
  • Fixed matmul accuracy when offset > 0 (#154495)
  • Fixed torch.xpu.is_bf16_supported to correctly report presence of Intel GPU (#152317)
  • Fixed AOT compilation in SYCL C++ extension (#156364)

Performance

Autograd
  • Improved autograd streams synchronization (#151079, #157914)
CPU (AArch64)
  • Compute ELU(0) with the cheaper definition (#155765)
CUDA
  • Improved performance of cat and index_select (#150233, #152380, #151715)
Dataloader Frontend
  • Reduced memory usage of SubsetRandomSampler by iterating over list instead of tensor (#149126)
torch.compile
Inductor
  • Improved performance of GEMMs (#147315, #151530, #149373, #156174, #155444)
  • Added a config option cpp.use_small_dequant_buffer to use a small dequant buffer for WOQ int4 GEMM (#156395)
  • Support graph partitioning on custom ops (#149782)
  • Optimized the heuristics of parallel reduction on CPU (#149614)
torch.export
  • Cache unflattened graph module (#150030)
JIT
  • Improved Dead Code Elimination (DCE) compile times for large graphs (#153645)
Linear Algebra Frontend
  • Introduced fast path for torch.dot with float16/bfloat16 (#152799)
MPS
  • Improved performance of LayerNorm, mm / bmm, sum / prod reductions, arithmetic ops, binary kernels, SDPA, linear, and cumsum / cumprod (#152010, #150541, #150566, #147644, #149730, #152781, #152210, #157494)
Python Frontend
  • Optimized SVE embedding performance (#150176)
  • Improved performance for torch.tensordot when contracting to a scalar (#145936)
ROCm
  • Improved performance of softmax, NLLLoss, in-place sum, max pooling backward / reductions on NHWC inputs, max pooling, multi-dimensional reductions, and non-vectorized elementwise kernels (#149076, #149779, #149548, #151230, #152267, #154522, #154619, #155806, #153184)
  • Improved scatter add performance on MI250X (#151724)
  • Extended vectorized elementwise kernel to more heterogenous tensor types (#149738)
  • Use HipSparseLT to further accelerate semi-structured (e.g. 2:4) sparsity (#150578)
Sparse Frontend
  • Skip sparse tensor invariant validation when loading sparse Tensors from external storage (#154610, #154759, #154638)
XPU
  • Enabled post-op fusion for oneDNN convolution on Intel GPU (#150287)
  • Reduced host overhead for Intel GPU by eliminating meaningless API calls (#151111)
  • Improved INT4 WOQ GEMM for Intel GPU by introducing a cache mechanism to reduce the oneDNN integration overhead further (#147693)
  • Improved scalar tensor case handling in addmm, baddmm to reduce oneDNN integration overhead on Intel GPU (#153051)

Documentation

Autograd
  • Added more details on why ctx.save_for_backward is important in note about extending autograd (#153005)
  • Updated docs of torch.autograd.graph.saved_tensors_hooks to avoid refcycle (#153049)
  • Updated gradient behavior note in torch.amin and torch.amax (#155071)
CUDA
  • Fixed deprecated amp APIs in docs (#154553)
  • Documented device memory apis in correct module (#155126)
  • Documented non-pytorch CUDA memory allocation and how to query it (#150880)
Distributed
c10d
  • Documented object collectives limitations (#150815)
  • Updated NCCLConfig with QOS variable (#151821)
  • Documented get_default_backend_for_device (#158236)
FullyShardedDataParallel2 (FSDP2)
  • Updated ignored_params docstring and added unit tests (#149074)
  • Added pointer to torchtitan (#153079)
  • Added warning for incorrected grad results at world size 1 (#154928)
torch.export
  • Added mini tutorial for provenance tracking (#152211)
  • Updated docs for Dims and ExportGraphSignature (#156262, #156244)
Linear Algebra Frontend
  • Addressed ambiguity in docs for torch.linalg.norm()'s ord argument of +2 & -2 (#155148)
torch.nn
  • Improved documentation for transformer-related layers, nn.RNN, nn.functional loss functions, interpolate saturate cast behavior, ConvTranspose2d stride / output_size arguments, and register_full_backward_hook (#155123, #153620, #148436, #151304, #150819, #150609, #151785)
  • Fixed examples for nn.Sequential and nn.LazyModuleMixin (#147304, #150596)
  • Documented padding size limitations in nn.modules.padding and AvgPoolND (#155618, #152680)
ONNX
  • Convert .rst doc files to markdown (#155228, #155556)
  • Improved docstring of ONNX symbolic ops (#149668)
  • Added note for attention op symbolic function (#156441)
  • Added ONNX Dynamo metadata documentation (#155816)
Optimizer
  • Added scripts to generate plots of LRSchedulers (#149189)
  • Included other accelerators in capturable docstr for optimizers (#149770)
  • Updated SGD documentation to match implementation and document that dampening is skipped in SGD first step (#149884, #152833)
  • Fixed doc for CosineAnnealingLR to accurately reflect its recursive learning rate schedule (#152936)
  • Fixed incorrect citation of authors in Adafactor documentation (#145209)
  • Added load_state_dict hint doc about invoke order work with lr_scheduler (#149942)
Python Frontend
  • Make torch.Library's kind have no default value to be consistent with the code (#149390)
  • Added 32-bit complex to the list of dtypes (#144590)
  • Clarified behavior when integer dtype is used with requires_grad=True in tensor.to() (#150913)
  • Optimized cdist param description (#151178)
  • Updated serialization docs (#153631)
  • Render Example: and not Example:: in docs (#153978)
  • Added docstring indicating undefined behavior for converting inf to int (#154781)
  • Updated as_strided() docs (#149146)
  • Fixed keepdim param optional description (#151197)
  • Clarify that x and dx are mutually exclusive in torch.trapezoid docs (#151190)
  • Documented out_dtype arg for torch GEMM operations (#151704)
  • Fixed the basic description of torch.min(), torch.max(), torch.all(), and torch.any() (#152658)
  • Added torch.triu_indices, torch.tril_indices dtype description (#150749)
  • Optimized torch.equal description (#149618)
Quantization
  • Fixed incorrect get_default_qat_qconfig in prepare_qat_fx docs (#155100)
Release Engineering
  • Migrated to new theme (#149331)
XPU
  • Improved "Getting Started on Intel GPU" hardware requirements and notes (#151886)

Developers

Distributed
c10d
  • Added param recording for uniqueID broadcasting and allgather (#149166)
  • Added logger config and more loggings, e.g. nccl_version and thread name/id, for flight record in PGNCCL (#150356, #150513, #151048, #152648, #155142, #155754)
  • Surfaced error type when we unlink and create named pipe for DumpPipe (#150648)
  • Improved the logs on remote shutdown of tcpstore (#153586)
  • Enhanced Error Logging in new_subgroups() for Non-Divisible World Sizes (#154124)
  • Added a logger for all nccl collectives with its time duration when completed (#156008)
  • Updated error message in get_backend() with more details (#141796)
FullyShardedDataParallel (FSDP1)
  • Print FQNs when debugging FlatParamHandle (#151336)
FullyShardedDataParallel2 (FSDP2)
  • Added FSDP2 logging (#155826)
RPC
  • Correctly pass exceptions raised from rpc_init to CPython (#154325)
torchelastic
  • Added the logging of start of torch elastic workers (#150849)
  • Passed event log handler to record function calls (#155457)
  • Added torch.distributed.run option to provide destination for event logging (#155268)
torch.export
  • Add TracingContext (#149294)
  • Monkeypatch fake mode so it errors on invalid custom ops (#149410)
  • Fixed torch export docs for preserve_module_call_signature (#151140)
  • Improved error message for deserializing custom triton op (#152029)
  • Better type annotation for lift_constants_pass (#152072)
  • Fixed bug in detect_attr_assignment (#151824)
Ahead-Of-Time Inductor (AOTI)
  • Refactor AOTInductor runtime API for Intel GPU (#153929)
  • Improve stable library APIs (#152040)
  • Add a basic shim and stable::Tensor is_contiguous API (#156228)
FX
  • Gracefully exit minimizer when there is no discrepancy in block mode (#154076)
Optimizer
  • Improve decorator typing for Optimizer subclasses (#153374)
  • Optimize typing in lr_scheduler.py (#151219)
  • Fixed the type hint of step() with default value (#153367)
Release Engineering
  • Added support for CUDA 12.9 in CI/CD (#154980, #156630, #155895, #155799, #155496, #155340, #155819, #156108)
  • Added support for ROCm 6.4 in CI/CD (#151236, #151345, #151355, #153253, #156112)
  • Moved CI from ubuntu 20.04 images to ubuntu 22.04 and 24.04 (#154437, #154153, #149142)
  • Moved CI to CUDA 12.8 (#154004, #152810, #155087, #148963)
  • Enabled CI on MI300 (#150667, #152133, #148394, #153134)
  • Enabled CI on H100 (#153900, #154562, #153170, #155861, #155719, #156429)
  • Enabled CD for Windows Arm64 (#150310, #152109, #149850, #152099)
  • Enabled testing of binary Docker builds in CI/CD (#151483, #151488, #151489, #151706)
  • Added smoke test to validate NCCL and cuDNN versions in PyPI packages (#149885, #150194)
  • Enabled monitoring for performance tests (#153452, #153453, #153454, #153456)
  • Improved benchmarking and performance testing on MacOS (#151721, #151747, #151748, #153897, #155493, #153897, #155493)
  • Use setup-python from for Mac tests (#155698)
  • Removed CUDA 11.8 and 12.4 support in CI/CD (#155509, #154169, #152362, #155555, #154893)
  • Removed Anaconda support in CI/CD (#147789, #152338, #152431, #152377, #152433, #147476, #151035, #152860, #152702, #154303, #154309)
View originalPermalink
v2.7.1

PyTorch 2.7.1 Release, bug fix release

This release is meant to fix the following issues (regressions / silent correctness):

Torch.compile

Fix Excessive cudagraph re-recording for HF LLM models (#152287) Fix torch.compile on some HuggingFace models (#151154) Fix crash due to Exception raised inside torch.autocast (#152503) Improve Error logging in torch.compile (#149831) Mark mutable custom operators as cacheable in torch.compile (#151194) Implement workaround for a graph break with older version einops (#153925) Fix an issue with tensor.view(dtype).copy_(...) (#151598)

Flex Attention

Fix assertion error due to inductor permuting inputs to flex attention (#151959) Fix performance regression on nanogpt speedrun (#152641)

Distributed

Fix extra CUDA context created by barrier (#149144) Fix an issue related to Distributed Fused Adam in Rocm/APEX when using nccl_ub feature (#150010) Add a workaround random hang in non-blocking API mode in NCCL 2.26 (#154055)

MacOS

Fix MacOS compilation error with Clang 17 (#151316) Fix binary kernels produce incorrect results when one of the tensor arguments is from a wrapped scalar on MPS devices (#152997)

Other

Improve PyTorch Wheel size due to introduction of addition of 128 bit vectorization (#148320) (#152396) Fix fmsub function definition (#152075) Fix Floating point exception in torch.mkldnn_max_pool2d (#151848) Fix abnormal inference output with XPU:1 device (#153067) Fix Illegal Instruction Caused by grid_sample on Windows (#152613) Fix ONNX decomposition does not preserve custom CompositeImplicitAutograd ops (#151826) Fix error with dynamic linking of libgomp library (#150084) Fix segfault in profiler with Python 3.13 (#153848)

View originalPermalink
v2.7.0

PyTorch 2.7.0 Release

PyTorch 2.7.0 Release Notes

Highlights

For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

Tracked Regressions

NCCL init hits CUDA failure 'invalid argument' on 12.2 driver

Some users with 12.2 CUDA driver (535 version) report seeing "CUDA driver error: invalid argument" during NCCL or Symmetric Memory initialization. This issue is currently under investigation, see #150852. If you use PyTorch from source, a known workaround is to rebuild PyTorch with CUDA 12.2 toolkit. Otherwise, you can try upgrading the CUDA driver on your system.

Backwards Incompatible Changes

Dropped support for Triton < 2.2.0. Removed Support for CUDA 12.4, Anaconda in CI/CD.
  • Removed CUDA 12.4 support in CI/CD in favor of 12.8 (#148895, #142856, #144118, #145566, #145844, #148602, #143076, #148717)
  • Removed Anaconda support in CI/CD (#144870, #145015, #147792)
  • Dropped support for Triton < 2.2.0 (versions without ASTSource) (#143817)
C++ Extensions py_limited_api=True is now built with -DPy_LIMITED_API (#145764)

We formally began respecting the py_limited_api=True kwarg in 2.6 and stopped linking libtorch_python.so when the flag was specified, as libtorch_python.so does not guarantee using APIs from from the stable Python limited API. In 2.7, we go further by specifying the -DPy_LIMITED_API flag which will enforce that the extension is buildable with the limited API. As a result of this enforcement, custom extensions that set py_limited_api=True but do not abide by the limited API may fail to build. For an example, see #152243.

This is strictly better behavior as it is sketchy to claim CPython agnosticism without enforcing with the flag. If you run into this issue, please ensure that the extension you are building does not use any APIs which are outside of the Python limited API, e.g., pybind.

Change torch.Tensor.new_tensor() to be on the given Tensor's device by default (#144958)

This function was always creating the new Tensor on the "cpu" device and will now use the same device as the current Tensor object. This behavior is now consistent with other .new_* methods.

Use Manylinux 2.28 and CXX11_ABI=1 for future released Linux wheel builds.

With Migration to manylinux_2_28 (AlmaLinux 8 based), we can no longer support OS distros with glibc2_26. These include popular Amazon Linux 2 and CentOS 7. (#143423, #146200, #148028, #148135, #148195, #148129)

torch.onnx.dynamo_export now uses the ExportedProgram logic path (#137296)

Users using the torch.onnx.dynamo_export API may see some ExportOptions become unsupported due to an internal switch to use torch.onnx.export(..., dynamo=True): diagnostic_options, fake_context and onnx_registry are removed/ignored by ExportOptions. Only dynamic_shapes is retained.

Users should move to use the dynamo=True option on torch.onnx.export as torch.onnx.dynamo_export is now deprecated. Leverage the dynamic_shapes argument in torch.onnx.export for specifying dynamic shapes on the model.

Version 2.6.0

torch.onnx.dynamo_export(model, *args, **kwargs)

Version 2.7.0

torch.onnx.export(model, args, kwargs=kwargs, dynamo=True)
Finish deprecation of LRScheduler.print_lr() along with the verbose kwarg to the LRScheduler constructor. (#147301)

Both APIs have been deprecated since 2.2. Please use LRScheduler.get_last_lr() to access the learning rate instead.print_lr and verbose were confusing, not properly documented and were little used, as described in #99270, so we deprecated them in 2.2. Now, we complete the deprecation by removing them completely. To access and print the learning rate of a LRScheduler:

Version 2.6.0

optim = ...
lrsched = torch.optim.lr_scheduler.ReduceLROnPlateau(optim, verbose=True)
// lrsched will internally call print_lr() and print the learning rate

Version 2.7.0

optim = ...
lrsched = torch.optim.lr_scheduler.ReduceLROnPlateau(optim)
print(lrsched.get_last_lr())
libtorch_python.so symbols are now invisible by default on all platforms except Apple (#142214)

Previously, the symbols in libtorch_python.so were exposed with default visibility. We have transitioned to being more intentional about what we expose as public symbols for our python API in C++. After #142214, public symbols will be marked explicitly while everything else will be hidden. Some extensions using private symbols will see linker failures with this change.

Please use torch.export.export instead of capture_pre_autograd_graph to export the model for pytorch 2 export quantization (#139505)

capture_pre_autograd_graph was a temporary API in torch.export. Since now we have a better longer term API: export available, we can deprecate it.

Version 2.6.0

from torch._export import capture_pre_autograd_graph
from torch.ao.quantization.quantize_pt2e import prepare_pt2e
from torch.ao.quantization.quantizer.xnnpack_quantizer import (
    XNNPACKQuantizer,
    get_symmetric_quantization_config,
)
quantizer = XNNPACKQuantizer().set_global(
    get_symmetric_quantization_config()
)
m = capture_pre_autograd_graph(m, *example_inputs)
m = prepare_pt2e(m, quantizer)

Version 2.7.0

from torch.export import export
from torch.ao.quantization.quantize_pt2e import prepare_pt2e
# please get xnnpack quantizer from executorch (https://github.com/pytorch/executorch/)
from executorch.backends.xnnpack.quantizer.xnnpack_quantizer import (
    XNNPACKQuantizer,
    get_symmetric_quantization_config,
)
quantizer = XNNPACKQuantizer().set_global(
    get_symmetric_quantization_config()
)
m = export(m, *example_inputs)
m = prepare_pt2e(m, quantizer)
New interface for torch.fx.passes.graph_transform_observer.GraphTransformObserver to enable Node Level provenance tracking (#144277)

We now track a mapping between the nodes in the pre-grad and post-grad graph. See the issue for an example frontend to visualize the transformations. To update your GraphTransformObserver subclasses, instead of overriding on_node_creation and on_node_erase, there are new functions get_node_creation_hook, get_node_erase_hook, get_node_replace_hook and get_deepcopy_hook. These are registered on the GraphModule member of the GraphTransformObserver upon entry and exit of a with block

Version 2.6.0

class MyPrintObserver(GraphTransformObserver):
    def on_node_creation(self, node: torch.fx.Node):
        print(node)

Version 2.7.0

class MyPrintObserver(GraphTransformObserver):
    def get_node_creation_hook(self):
        def hook(node: torch.fx.Node):
            print(node)
        return hook
torch.ao.quantization.pt2e.graph_utils.get_control_flow_submodules is no longer public (#141612)

We are planning to make all functions under torch.ao.quantization.pt2e.graph_utils private. This update marks get_control_flow_submodules as a private API. If you have to or want to continue using get_control_flow_submodules, please make a private call by using _get_control_flow_submodules.

Example: Version 2.6:

>>> from torch.ao.quantization.pt2e.graph_utils import get_control_flow_submodules

Version 2.7:

>>> from torch.ao.quantization.pt2e.graph_utils import get_control_flow_submodules
ImportError: cannot import name 'get_control_flow_submodules' from 'torch.ao.quantization.pt2e.graph_utils'
>>> from torch.ao.quantization.pt2e.graph_utils import _get_control_flow_submodules  # Note: Use _get_control_flow_submodules for private access

Deprecations

torch.onnx.dynamo_export is deprecated (#146425, #146639, #146923)

Users should use the dynamo=True option on torch.onnx.export.

Version 2.6.0

torch.onnx.dynamo_export(model, *args, **kwargs)

Version 2.7.0

torch.onnx.export(model, args, kwargs=kwargs, dynamo=True)
XNNPACKQuantizer is deprecated in PyTorch and moved to ExecuTorch, please use it from executorch.backends.xnnpack.quantizer.xnnpack_quantizer instead of torch.ao.quantization.quantizer.xnnpack_quantizer. (#144940)

XNNPACKQuantizer is a quantizer for xnnpack that was added into pytorch/pytorch for initial development. However, as it is not related to our core quantization workflow, we have moved it to ExecuTorch instead. Please use it from executorch.backends.xnnpack.quantizer.xnnpack_quantizer instead of torch.ao.quantization.quantizer.xnnpack_quantizer.

Version 2.6.0

from torch._export import capture_pre_autograd_graph
from torch.ao.quantization.quantize_pt2e import prepare_pt2e
from torch.ao.quantization.quantizer.xnnpack_quantizer import (
    XNNPACKQuantizer,
    get_symmetric_quantization_config,
)
quantizer = XNNPACKQuantizer().set_global(
    get_symmetric_quantization_config()
)
m = capture_pre_autograd_graph(m, *example_inputs)
m = prepare_pt2e(m, quantizer)

Version 2.7.0

# we also updated the export call
from torch.export import export
from torch.ao.quantization.quantize_pt2e import prepare_pt2e
# please get xnnpack quantizer from executorch (https://github.com/pytorch/executorch/)
from executorch.backends.xnnpack.quantizer.xnnpack_quantizer import (
    XNNPACKQuantizer,
    get_symmetric_quantization_config,
)
quantizer = XNNPACKQuantizer().set_global(
    get_symmetric_quantization_config()
)
m = export(m, *example_inputs)
m = prepare_pt2e(m, quantizer)

New features

Release Engineering
  • Added support for CUDA 12.8 in CI/CD (#145567, #145789, #145792, #145765, #146019, #146378, #146957, #147037, #146265, #147607, #148000, #149584)
  • Added Python 3.13 and 3.13t support in CI/CD (#144698, #143078, #144697, #143074, #141806, #146614)
  • Added aarch64 support for pytorch-triton package (#148768, #148705)
  • Added support Windows XPU CI/CD (#148755, #147637, #148313, #143185, #148319, #144316, #144644, #144034, #145255)
  • Added support for ROCm MI300 CI/CD (#143673, #145504, #146675, #147904, #145398, #145621, #145829, #145790, #144594)
  • Added support for PEP585, Type Hinting Generics In Standard Collections (#145707, #145177, #145708, #145342, #145101)
  • Added Windows Arm64 Nightly Builds (#139760)
Python Frontend
  • Introduce a new torch.utils.serialization.config namespace for all serialization related configurations (#143324)
  • Add torch.serialization.config.save.use_pinned_memory_for_d2h to speed up torch.save when passed gpu devices (#143342)
  • Add torch.utils.serialization.config.load.calculate_storage_offsets to reduce random reads and significantly improve performance for storage with bad random access performance (#143880)
  • Add support for __torch_function__ handler on dtype arguments, similar to subclass objects (#145085)
C++ Extensions
  • Support libtorch-agnostic extensions with stable torch ABI (#148892, #148832, #148124, #149208, #149052)
Distributed
Context Parallel
  • We provided a Context Parallel API (#131351) for users to parallelize torch.nn.functional.scaled_dot_product_attention over the sequence dimension. We implemented Ring Attention (#131351) and an AllGather-based approach (#132820) where the all-gather is issued before the first local SDPA and the subsequent local SDPAs will have to wait until the all-gather completes, and offered a user API (#142093) to select the desired approach. The implementation currently supports three SDPA kernels: SDPBackend.FLASH_ATTENTION, SDPBackend.EFFICIENT_ATTENTION, and SDPBackend.CUDNN_ATTENTION (#148537). We also verified that our Context Parallel implementation is compatible with other parallelisms and torch.compile.
c10d
  • Implemented ncclCommInitRankScalable (merging #136789) (#144794)
Distributed Checkpoint (DCP)
  • Cache save plans: to mitigate overhead from planning steps (#147116, #147343)
  • Build a storage reader/writer to write checkpoints in HF format (#148089)
CUDA
  • Blackwell support added across native kernels, CUDA math libraries, and torch.compile (#145270)
  • Make torch.cuda.gds APIs public (#147120)
MPS
  • Prototype of torch.compile for Metal (#143893)
  • Provide Metal kernel authoring via Python (#148972)
ROCm
  • CK Memory-Efficient Attention (attention bias support) (#147778)
  • CK Flash Attention Backend (#143695)
  • Enhanced Windows support for PyTorch on ROCm (#148563, #144098)
  • Support for gfx1102 arch (Navi33) in wheel builds (#147761)
  • hipblaslt rowwise f8 gemm (#144432)
XPU
  • Add AOT Inductor support for Intel GPU (#140269, #140664, #149175)
  • Support torch.compile on Windows Platform for XPU (#147637, #144316, #149511)
  • Support SYCL with torch.utils.cpp_extension APIs (#132945)
  • Enhance Intel GPU performance on PyTorch 2 Export Post Training Quantization (#136753, #135465,#135337, #135189)
  • Enable windows Kineto profiler(#148319)
  • Enable TF32 support for XPU based on oneDNN backend (#137570)
torch.compile
Dynamo
  • Support tracing contextlib.contextmanager in Dynamo (#136033)
  • nonstrict_trace escape hatch to apply non-strict tracing to difficult-to-compile code (#146367)
  • Delayed compile for dynamic shapes (#147983)
  • Support tracing generators (#141055)
  • Whitelist of source files to apply dynamic shapes to (#147979)
  • Support tracing list subclasses (#146819)
Inductor
  • Enable non power-of-2 head_dim for FlexAttention (#133495).
  • Add FlexAttention kernel parameter tuning options: num_warps and num_stages (#139639).
  • Support vectorization for score and mask in FlexAttention CPU (#143638).
  • ConfigFuzzer: a new debugging tool designed to fuzz Torch compile configurations. Given a test function, it will identify combinations of configs that throw errors during compilation and execution (#139736) (#145565).
  • Support fusion of pointwise ops into Template Prologues. TORCHINDUCTOR_PROLOGUE_FUSION enables this feature (#147008).
  • Add instantiation level for generating configs in the CUTLASS backend. Set TORCHINDUCTOR_CUTLASS_INSTANTIATION_LEVEL. Consult config.py for information (#146230).
  • Add L2 Swizzle config for CUTLASS backend: cuda.cutlass_max_profiling_swizzle_options (#146088).
  • Emit a CMakeLists.txt when package_cpp_only is specified in AOTI (#143352).
  • One Dynamo graph can now map to multiple inductor graphs with different graph_partition functions. Set the graph_partition in inductor config to enable (#147038).
Profiler
  • Add overload names to profiler (#143114)
  • Enable profiling on all threads via experimentalConfig (#143659)
Quantization
  • Enables kernel from KleidAI to run model that was quantized such that weights are in int4 (with symmetric quantization either using channel-wise or group-wise, with the group size being a multiple of 32), while at runtime the activations are dynamically quantized from fp32 to int8 and weights are upcast from int4 to int8 so that int8 matrix multiplication is executed. This dynamic quantization of activations and matrix multiplication is performed inside of function torch.ops.aten._dyn_quant_matmul_4bit, while the weights, scaled and optional bias are packed in torch.ops.aten._dyn_quant_pack_4bit_weight. To use it on your model you can quantize it using the following example that leverages torchao:
from torchao.dtypes import PlainLayout
from torchao.experimental.packed_linear_int8_dynamic_activation_intx_weight_layout import (
    PackedLinearInt8DynamicActivationIntxWeightLayout,
)
from torchao.experimental.quant_api import (
    int8_dynamic_activation_intx_weight,
)
from torchao.quantization.granularity import (
    PerGroup,
    PerRow,
)
from torchao.quantization.quant_api import quantize_
from torchao.quantization.quant_primitives import MappingType
my_model = Model()
quantize_(
    my_model,
    int8_dynamic_activation_intx_weight(
        weight_dtype=torch.int4,
        granularity=PerGroup(32), # PerRow() is also supported
        has_weight_zeros=True, # Should be True
        weight_mapping_type=MappingType.SYMMETRIC_NO_CLIPPING_ERR # MappingType.SYMMETRIC can also be used but increases error
        layout=PackedLinearInt8DynamicActivationIntxWeightLayout(target="aten"),
    ),
)
ONNX
torch.onnx.verification.verify_onnx_program (#148396, #148706, #148730, #148707)

A new verification API torch.onnx.verification.verify_onnx_program can now be used to verify numerical accuracy of the exported ONNX model. Users can use the compare_intermediates option to identify any operator that causes numerical discrepancies in intermediate tensors. It is possible to use a tool like model-explorer to visualize the verification results.

  • Support custom axis name through dynamic_shapes (#146321)
  • torch.onnx.export(dynamo=True) now optimizes the output model by default (#146187)

Improvements

Release Engineering
  • Added TorchCache Benchmark tests (#147641, #147688, #147782, #147780, #147781, #147783, #147546)
  • Upgrade CD to 6.3 for ROCm (#142152, #142151, #143613)
  • Add cufile to a dependency list for CUDA 12.x builds and enable use by default (#145748, #148465, #148137)
  • Add support for gfx1102 and gfx12 to ROCm wheel and libtorch builds (#147761, #148562)
Python Frontend
  • Add support for CPU scalar in torch.addcmul (#143264)
  • Set -DPy_LIMITED_API flag for py_limited_api=True cpp_extensions (#145764)
  • Add support for serialization for uintx/intx in weights_only (#147500)
  • Add warning to torch.jit.load (#143403)
  • Make record/storage alignment in torch.save configurable (#147788)
  • Support with statement on torch.Stream (#140138)
Autograd
  • Allow torch.autograd.graph.GradientEdge as torch.autograd.backward outputs #144744
  • Implement gradient for the residuals of torch.linalg.lstsq #148526
  • Add deterministic kernel for reflection_pad2d_backward (#136241)
  • Improve softmax backward pass native CUDA implementation (#145866)
  • Improve Pareto frontier plot for AutoAC (#148678)
Dataloader
  • Dataloader distributes tasks to workers as they become available when in_order is False (#142324)
  • Update pin memory related APIs to not pass device argument. device and pin_memory_device are discouraged and will be deprecated in the future. (#131858)
Linear Algebra
  • Improve dim argument validation for empty inputs for torch.cum{min,max}. (#143920)
  • Properly throw an error when trying to sort complex numbers. (#144113)
Nested Tensor (NJT)
  • Support NJT chunk() backward on batch dim (#144584)
  • Support remaining *_like factory functions for NJT (#144889)
  • Improve matmul with NJTs via backward support and composition with dense tensors (#144587, #146405)
torch.nn
  • Add strict kwarg to nn.Module.set_submodule and fix bug for non dot-delineated strings (#143455)
  • Improve input dimensions check for reflection_pad1d, reflection_pad2d and reflection_pad3d (#141670)
torch.optim
  • Refactor AdamW to subclass Adam (#143710, #144972)
  • Add support for differentiable LR and weight_decay in SGD, Adam(W) (#143510, #143679, #143726)
Build Frontend
  • Make PyTorch with HomeBrew installed OpenMP (#145870)
  • Enable onednn in pytorch for ppc64le architecture (#143743)
  • Enable build for Blackwell GPU family (#145436)
  • Fix OOM whle building on RasberryPi by sharding codegenerated files (#144364)
C++ Frontend
  • Introduce a new API isAcceleratorExcluded (#144959)
Distributed
c10d
  • Simplified abort and shutdown by adding both to Backend and ProcessGroup objects (#148798)
  • Used new_group instead of split_group on non-CUDA device (#141469)
  • Removed call_guard in pybind object init of c10d (#143598)
  • Enabled coalescing path on XPU and dispatch to XPU tensor barrier if XCCL backend is specified. (#143735)
  • Preserved PyWork's Python reference counting when used in functional collectives (#146376)
  • Enabled soft fail bind when agent store active inside TCPStore (#147465)
  • Made getDefaultBackend more fault tolerant (#148596)
DistributedDataParallel (DDP)
  • Added init_sync option to control collectives during initialization (#142824)
  • Decoupled python reducer from compilation mode (#147123)
FullyShardedDataParallel2 (FSDP2)
  • Clamp reduce_dtype in lazy init (#143297)
  • Enabled FSDP2 on XPU device (#143737)
  • Made post-backward condition more robust (#144781)
  • Enabled MTIA device in FSDP2 library code (#145842)
  • Avoided resetting version counter of all_gather_output in inference_mode (#146709)
  • Supported ignoring parameters in FSDP2 (#146631)
  • Enabled FSDP tests on XPU device (#147518)
  • Enabled FSDP2 on HPU device (#148667)
DTensor
  • Added aten.amin/amax to linear_reduction_strategy (#143747)
  • Added src_data_rank to distribute_tensor API (#143883)
  • Added strategy for _scaled_mm (#143760)
  • Added aten.view.dtype op support (#144404)
  • Enabled sharding prop to handle cross mesh computation (#147869)
  • Added CuDNN SDPA op support to DTensor (#148537)
  • Optimized shard_dim_alltoall to use alltoall_single (#148868)
  • Deprecated _shard_tensor to use src_data_rank=None (#144171)
  • Added pointwise ops strategy for aten.minimum (#145816)
TensorParallel
  • Propagated src_data_rank kwarg in TP API (#144005)
Torch Elastic
  • Added kill logic for current process when killing a worker (#141060)
  • Made etcd_rendezvous publicly importable (#145396)
  • Exposed the rendezvous keepalive arguments (#145228)
Pipelining
  • Added generate_stage_to_rank_mapping utility (#146193)
  • Removed stage_index_to_group_rank from schedule (#146217)
CPU
General
  • Implement blend operation for float, double, int in VEC ATen backend for SVE (#146479)
  • Upgrade submodule oneDNN to v3.7.1 (#148293)
x86
  • Add support for int8 brgemm (#143384)
CUDA
  • Refine CUDA Stream priority (#143849)
  • Expose sharedMemPerMultiprocessor device property to python (#143119)
  • Expose remaining sharedMem cudaDeviceProps to python (#143226)
  • Add range check for embedding_bag on input index >= 0 of cuda device (#140791)
  • Fix linter warnings (#147386)
  • Change behavior of pinning memory so it does not init a cuda context if one is not already present (#145752, #149033)
  • Add cutlass kernel for rowwise scaled mm on SM 10.0 (blackwell) (#148421)
  • Add get_stream_from_external API for CUDA backend (#143799)
  • Update cuDNN-frontend submodule to 1.10.0, used by cuDNN convolution and SDPA integrations (#145780)
MPS
  • Adding support to MPS for operators: angle, entr, spherical_bessel_j0,xlog1py, sinc,round.decimals, linalg.det, cholesky.ex, bilineard2d_aa,linalg.solve, zeta, cholesky, fused_rms_norm, lu_unpack, lu_factor_ex, slogdet and logdet (#143449, #147948, #146818, #147687, #146539, #147266, #146279, #146799, #145526, #146531, #146465, #145701, #145301, #146681, #144651, #145341, #146771, #147914)
  • Extending data type support for angle and atan2 for long type, torch.special.sinc to complex, torch.mm / torch.bmm to integral types (#149017, #146648, #145809, #147526)
  • Support torch.accelerator.synchronize() on MPS (#143171)
  • Add error checking when dispatching kernel (#146458)
  • For MPSInductor
    • Fix index generation for transpose (#143973)
    • Fix multi rangevar kernel invocation (#144050)
    • Better error when kernel fails to compile (#144649)
    • Fix large prod and sum reductions (#148975)
    • Adding support to MPSInductor for operators: gamma, zeta, sinc, spherical_bessel_j0, entr (#145341, #146465, #146539, #147650, #148128)
ROCm
  • Fix TunableOp UTs: Rotating Buffer (#143172)
  • Enable *_load_dwordx4 ISA for BFloat16 and Half. (#141397)
  • Fix condition for small tensor tuning (#144087)
XPU
  • Enable FP64 GEMM (#140677)
  • Enable Sparse CSR support (#144722)
  • Improve XPU Stream implemenation(#141123,#141119,#142347)
  • Enable XPU for Inductor MM Triton Kernel Benchmark (#148237)
  • Align XPU convolution_backward output layout between fake tensor and real output tensor (#146880)
  • Improve error handling and reporting in CMake files (#149353)
  • Refine torch.xpu.get_device_properties API error message (#144379)
  • Enable nested_layer_norm support for XPU (#148593)
  • Generalize is_big_gpu() check in Inductor (#143491)
  • Allow XPU device in sparse compressed tensor factory functions (#147306)
Profiler
  • Add optional flag to profiler to toggle external correlations (#143314)
  • Add delimeter in memory vizualizer to show where allocation addr begins (#147461)
  • Add last entry to truncated values in Kineto args (#148576)
  • Add profiler activity for HPU devices (#148182)
  • Add HPU availabilities to profiler (#149115)
torch.compile
Dynamo
  • Better tracing support for user-defined dict subclasses (#143548)
  • Improved graph break messages for some common graph break sites (#146525)
  • Improved tracing of exceptions (#146492)
  • Remove a number of builtin and third-party modules from trace_rules.py skipfiles (#145856)
  • Remove some specialized variables for specific third-party classes (e.g. transformers ModelOutput) (#143567)
  • Compiled Autograd dropped annotation requirements for custom autograd functions (#146229, #146720)
AOTDispatcher
  • Fix a quadratic compile time edge case during training when you have long parallel chains of compute (#145082)
  • handle compiling mutations on tangents in custom autograd.Functions (#141131)
  • handle compiling buffer input mutations of the form buffer.copy_(int) (#141161)
  • Fix handling of mutable custom operators in compile when used with torch.inference_mode (#147925)
Dynamic Shapes
  • Better unbacked symint handling for topk (#147017)
  • dynamic shape support for interpolate(antialias=True) backward (#141198)
  • Better unbacked symint handling in the partitioner (#143877)
  • Support dynamic shape inputs to nonzer_static (#146006)
  • Improve logging in the symbolic shapes framework (provenance tracking, error messages) (#143378, #146625, #146583, #146532, #145354, #146858, #146939, #146955, #147240,#146413m #145848, #147836, #146298)
  • Simplify and speed up _compute_symbolic_stride() (#138844)
  • Add max kwarg to torch._check (#144471)
  • Apply hints to symbol not expr when materializing unbacked tensor intermediates in the partitioner (#144097)
  • Add backed_size_oblivious config (#148696)
  • Add mark_unbacked strict mode (#147333, #147342)
Decompositions, FakeTensor and meta tensors

Several operator decomps received improvements/bugfixes:

  • torch._refs.tensor (#143461)
  • torch._refs.mean (#147188)
  • linspace (#147997)
  • addmv (#143792) New meta tensor implementations for a few pytorch operators:
  • nonzero (#144727)
  • silu, sigmoid, _softmax, embedding (#147862) New fake tensor implementation for a few pytorch operators:
  • unique_consecutive (#145649) Several general FakeTensor improvements
  • force UntypedStorage.from_buffer(buf) to return meta storage under FakeTensorMode (#146642)
  • support meta_tensor.to(device='cpu') under fake_mode (#146729)
Inductor
  • Add profiling support for codegened CPU FlexAttention kernels (#145894).
  • Other FlexAttention improvements: (#147765) (#147435) (#147010) (#146657) (#145059) (#144938) (#143299) (#142281) (#147918) (#148857).
  • Add Inductor support for non-power-of-2 cooperative RSPLIT (#145689).
  • Remove runtime dependency on packaging (#149125)
  • Add Cutlass support for runtime param choices, starting with swizzle (#147223).
  • Make Inductor cpp backend enable_floating_point_contract_flag take string. Previously, the only options were "on" or "off". Now the value of INDUCTOR_CPP_ENABLE_FLOATING_POINT_CONTRACT_FLAG will be passed to ffp-contract (#143450).
  • Add upcasting FP16/BF16 math reductions to FP32 in Triton (#141052).
  • Support for more types of async_compile pools. Set variable TORCHINDUCTOR_WORKER_START to one of "subprocess", "fork", or "spawn" (#144491).
  • Create a new benchmarker to replace Triton's do_bench (#133058).
  • Inplace-padding support for cpp-wrapper (#145325).
  • New environment variables for emulate_precision_casts: TORCHINDUCTOR_EMULATE_PRECISION_CASTS (#145948).
  • New environment variables to filter cutlass kernels: TORCHINDUCTOR_CUTLASS_ALLOWLIST and TORCHINDUCTOR_CUTLASS_DENYLIST (#148161).
  • Add option to disable runtime scalar assertions: TORCHINDUCTOR_SCALAR_ASSERTS (#146462).
  • Add new inductor configs to compiler bisector: layout_optimization and comprehensive_padding (#148450).
  • Add an option to skip optimizing generated wrapper code. Set AOT_INDUCTOR_COMPILE_WRAPPER_WITH_O0=1 (#144866).
  • Support dynamic shape constraints in Export (#146044).
  • Handle MLIR scf.yield more accurately in user Triton code (#147762).
  • Support Triton 3.3: add a global_scratch arg, fix cpp_wrapper (#148051, #149973).
  • Removed an unnecessarily struct runtime alignment assertion, allowing more flexible use cases of AOTI (#143236).
  • Support _int_mm in AOTI (#144571).
  • Support AOTI + CUDAGraphs when calling from Python (#148601).
  • New post grad pass to remove torch.ops.aten._assert_tensor_metadata.default for AOTI (#145028).
  • Support basic TorchBind in aot_compile and aoti_compile_and_package (#148506).
  • Add top level tlparse logging for AOTI (#147760)
  • Added Inductor dashboard benchmarks (#144427, #145791, #145654, #145655, #146449, #145683, #141371, #143223)
  • Add AOTI shim for _weight_int4pack_mm_cpu_tensor (#149031)
torch.fx
  • Fix subgraph rewriter to support matched pattern with no users (#143842)
  • Improve error message to include entire GraphModule (#146197, #148090)
  • Allow overriding of ShapeProp (#148784)
torch.export
serialization
  • Add float8 support in serialization schema (#143343)
  • Allow pickle protocol overriding for serialization (#142253)
  • Add serialization support for SymInt inputs in higher-order op subgraphs (#142284)
  • Unify single-output and multi-output serialization schemas for higher-order op subgraphs (#143227)
  • Add "+export" logging to de/serialization process (#145283)
  • Sync model container types to serialization schema (#145959)
  • Serialize pytree namedtuple field names in input spec (#145956)
  • Replace builtins.getattr with serializable higher-order-op for tensor subclasses (#145772)
dynamic shapes
  • Support slice operations with SymInt indices in non-strict export (#143217)
  • Export with automatic dynamic shapes (Dim.AUTO) for TorchScript -> Export Converter (#138273)
  • Support partially specifying dimensions in ShapesCollection (#147534)
draft export
  • Report frequency of data-dependent errors in draft export (#145030)
  • Report LOC for data-dependent errors in draft export (#145443)
  • Add tlparse for draft export (#145810)
  • Deduplicate expression_created logging in draft export (#146859)
  • Remove report as return output for draft export, attached as ep._report (#147558)
miscellaneous
  • Don't decompose custom triton ops when exporting (#144284)
  • Handle input/buffer mutations for joint-graph export (#144806)
  • Allow builtin bitshift ops in verifier (#145802)
  • Introduce aoti_call_delegate higher-order-op for eager-mode runnability (#145630)
  • Include tensor subclass buffers in parametrization rules (#145991)
  • Expose pytree namedtuple metadata to FlatArgsAdapter (#146107)
  • Implement OSS-only model runner (#146440)
  • Exclude core ATen ops upsample_bilinear2d.vec, nearest2d.vec from default decomposition table (#147153)
  • Improve error message for unsupported input types (#147532)
  • Initial support for exporting methods (#147573)
Quantization
  • Add an option keep_original_weights in _lower_to_native_backend (#141049)
  • Handle meta tensors in FX quantization (#144726)
  • Add fp8 support to index_cuda (#144747)
  • Add the torch.float8_e8m0fnu dtype to PyTorch (#147466)
  • Improve the performance of 8 bit quantized linear and addition operation on AArch64 by leveraging operations from Arm Compute Library (#148585)
  • Enables int8 linear operations to use mkl-dnn when activations, weights and accumulation are all signed 8-bit integers (#139887)
ONNX
  • Dynamic shapes support is improved (#144801)
  • Automatically convert dynamic_axes to dynamic_shapes with torch.export.Dim.AUTO (#143158)
  • Fix bug for exporting torch.cdist into onnx and support 'compute_mode' (#144213)
  • Remove LegacyDynamoStrategy (#145442)
  • Set warning stacklevel so it appears at the torch.onnx call site (#147165)
  • Pick up missing types in dynamic_shapes renaming (#147407)
  • Update saved exported program in debugging report if the exporting passes run_decomposition() (#148617)
  • Use torch export to get dynamic_shapes for JIT convert strategy (#148627)
  • Use torch.export.Dim.AUTO in dynamo_export (#144356)
  • Support complex comparison when verify=True (#148619)
JIT
  • Relax type-checks for empty dicts (#147167)
Lazy Tensor
  • Introduce cache clearing APIs for the lazy graph executor (#144489)
torch.package
  • Add support for UntypedStorage tensors (#143930)

Bug fixes

Python Frontend
  • Fix torch.lerp type promotion (#141117)
  • Fix memory leak on torch.Tensor when both slots and python gc are used (#143203)
  • Fix torch.bfloat16 support for __cuda_array_interface__. (#143042)
  • Fix rare dispatcher bug for inplace operations that would make the returned torch.Tensor incorrect. (#145530)
  • Stop using MKL for randomness generation on CPU (#146174)
  • Move accelerator detection to use build time (#146098)
  • Fix torch.load under FakeTensorMode to create FakeTensor with correct devices (for plain Tensors) (#147786)
  • Fix torch.acos, torch.asin, torch.atan, torch.exp, torch.sigmoid, torch.div, for torch.complex datatypes on CPU (#134838, #140358, #140391, #140375, #144749)
Autograd
  • Fix torch.autograd.graph.allow_mutation_on_saved_tensors for inplace foreach ops #145520
  • Fix boundary conditions for hardswish backward (#143899)
  • Use float data type for Half sum in fallback implementation of batchnorm backward on CPU (#147353)
  • Fix torch.compile + ddp + non-reentrant AC pack hook firing count (#144271)
Linear Algebra
  • Fix workarea compute in eigh (#146456)
Nested Tensor (NJT)
  • Fix NJT min / max backward() for non-ragged reductions (#144583)
  • Fix NJT frexp() to handle both outputs (#144585)
  • Fix NJT fill.Scalar for contiguous inputs (#144586)
  • Fix inference mode for composite implicit ops without nested-specific kernel (#146633)
  • Fix flop counter for SDPA and test (#147032)
torch.nn
  • Fix broken meta function for flex-attention backwards (#146563)
Build Frontend
  • Fix unbalanced #pragma diagnostic pop in VecLib (#148354)
  • Fix atomic operation compatibility for ARMv8-A (Raspberry Pi 4) by adjusting compilation flags (#148070)
  • Make PyTorch buildable by CMake-4.x (#150203)
C++ Frontend
  • Fix Apple Clang ICE when building with -march=armv8.6a (#142879)
  • Fix inductor regression on aarch64 neoverse-v1 with gcc10.2 by disabling tree vectorization (#148489)
Distributed
Distibuted Checkpoint (DCP)
  • fix dcp gather_object/scatter_object_list (#147675)
Distributed (c10d)
  • Fixed CudaEventCache for dangling references (#144496)
  • Make all-reduce input contiguous in distributed.nn.all_reduce (#144267)
  • Removed Alltoallv specialization for PyTorch generic all_to_all (#145045)
  • Added a handle case when remote peer closes connection for TCPStore (#145757)
  • Fixed memory leak on shutdown (#145507)
  • Fixed an issue where functional collectives don't force fx stride on inputs when compiled (#146467)
  • Associated tensor allocation support with NCCL version (#146842)
  • Modified API to get device string from device with torch.device (#146290)
  • Fixed dist.init_process_group on windows (#148266)
  • Fixed capturability of isend and irecv (#148462)
DistributedStateDict (DSD)
  • Fixed strict=False case for DDP (#143038)
  • Fixed issue when there is a PG without parameters (#147730)
  • Fixed the shared parameter mismatch for optimizer state_dict when flattening FQNs are used (#148825)
FullyShardedDataParallel2 (FSDP2)
  • Rooted fix for FP8 tensor (#143248)
  • Added workaround to fix buffer_dtype without root parameters (#143989)
  • Supported custom all reduce hook across FSDP units (#147114)
  • Fixed bug in FSDP wrapped module with zero argument (#147771)
DTensor
  • Fixed torch.distributed._functional_collectives.AsyncCollectiveTensor for aten.to. (#134661)
  • Deferred DTensor RNG state sync until first random op call or manual_seed call to support more flexible OffsetBasedRNGTracker init (#147025)
  • Fixed _scaled_dot_product_flash_attention sharding (#148125)
  • Fixed redistribution cost for all-reduce (#148761)
Pipelining
  • Fixed backward_one_chunk when the output of the model is a view (#142237)
  • Threw error with ZB and compile (#143599)
  • Fixed FSDP+PP stream sync bug (#144535)
  • Fixed PP grad scaling (#144352)
  • No allowing for num_microbatches > num_stages for single stage schedules (#144702)
  • Fixed shape_inference for V-schedules (#147000)
CPU
General
  • Use sleef implementation for CPP backend asinh codegen (#142360)
x86
  • Constrain the shape of other tensor for Conv/Linear + broadcast add fusion (#141759)
CUDA
  • Let PYTORCH_NO_CUDA_MEMORY_CACHING has effect only when value is 1 (#145905)
  • Fix race condition in cuda initialization (#143238)
  • Fix a few 64-bit indexing issues, account for number of threads in complex128 scan (#143401)
  • Fix acquire pattern (correctness with respect to memory model) in topk (#144945)
  • Int64 indexing fix for UpSampleNearest3D (#144865)
  • Fix printing of the number of GPUs when certain asserts are raised (#146838)
  • Update the number of threads in avg_pool2d backward for SM 10.0 to prevent runtime crash (#145669)
  • Only use f8f8bf16 rowwise scaled matmul to SM 9.0 (precedes #148421 adding of kernel) (#145728)
  • Fix 64-bit indexing for Upsample2D (#141923)
  • Fix path lookup in _preload_cuda_deps (#149808)
  • Help support Blackwell: Fix backward launch bounds again for sm100, sm120 (#150640)
MPS
  • Workaround for gather_out in MPS backend (#135543)
  • Fix fmin/fmax for scalar argument (#143934)
  • Fix crash when mm is invoked with mixed dtypes (#143948)
  • Fix torch.add(x,y, alpha=2) crash (#143949)
  • Fix nllnd_loss_backward crash with different dtypes (#144170)
  • Make sure that MPSStream is usable from C++ (#144559)
  • Make MPSProfiler usable from C++ (#144560)
  • Fix regression in con-contiguous bitwise ops (#146085)
  • Fix lu factor for large tensors with bs>1 (#146753)
  • Ensure 4d input in _scaled_dot_product_attention_math_mps (#146623)
  • Fix cholesky_ex for empty inputs (#147159)
  • Fix attention for >4d tensors (#147545)
  • Fix empty placeholder error for smooth l1 loss (#148133)
  • Fix sqrt and other for torch.chalf (#148285)
  • Fix unary_kernel_strided logic (#148512)
  • Fix scalar to tensors bitshifts (#148686)
  • Fix multinomial sampling for non-contiguous tensors (#141515)
  • Fix triangular for >3D tensors (#144545)
  • Fix missing autorelease in lstm_mps causing leaked memory (#145503)
  • Fix missing autoreleasepool around runUniqueGraph to prevent leaks (#145512)
  • Workaround rng bug for 5D tensors (#147667)
  • Fix Wreorder-init-list (#148839)
  • Fix invalid format string in libfmt calls (#148855)
  • Fix c10::metal::log_gamma correctness on M4 (#145740)
  • Fix lu factor for non contiguous tensors (#146279)
  • Fix attention enable_gqa crash on MPS (#149147)
  • Fix dot/mm for conj_tensors (#150157)
  • Fix tril op not handling infs correctly (#149866)
  • In MPSInductor:
    • Fix min/max reductions over large dims (#149004)
    • Fix argmin/max signatures (#149020)
    • Fix masked/where for inf values (#144500)
    • Move threadfence to before first read from shared memory, not after (#149437)
ROCm
  • TunableOp use thread-safe getenv functions (#142274)
  • fix torch.layer_norm invalid configuration problem when input is large tensor (#144007)
  • [Inductor][CK] hackfix for segfault in addmm op (#144519)
  • Fix torch.layer_norm invalid configuration when input is large tensor (#144007)
  • Fix isnan integer overload errors on MicroSoft STL (#146605)
  • Fixes and improvements to CUDA->HIP flag conversion for CPP extensions (#149245)
XPU
  • Fix SDPA dummy log_sum_exmp output to match meta function (#148652)
  • Fix memory leak in deconv backward (#144385)
  • Add XPU support to torch.utils._content_store to accelerate XPU tensor hashing for tensor serialization (#147785)
  • Enabling XPU in OffsetBasedRNGTracker to unbreak torch.distributed (#148360)
  • torch.backends.mkldnn.flags() CM should not warn (#150358)
Profiler
  • Hide Kineto step() for iterative on-demand tracking behind environment variable (#144494)
  • Enable CUPTI on Windows (#141454)
  • Fix device setting error of other backends in torch.profiler (#144237)
  • Fix assertion failure in PyTorch profiler (#143940)
torch.compile
  • Do not depend on numpy during torch._functorch import (#149683)
Dynamo
  • Guard on global autocast state (#143592)
  • Fix some internal crashes involving undefined names (#144784)
  • Multiple silent incorrectness fixes for Compiled Autograd (#144707)
  • Fix graph break in FlexAttention when using Compiled Autograd (#144533)
Inductor
  • Fix a bug where the options dictionary on torch.compile calls was ignored (#145131).
  • Inductor now supports nanj in cpp wrapper CPU (#144064).
  • Fix a bug in the fractional_max_pool lowering in Inductor (#144395).
  • FlexAttention: Fix a few more symbolic shape issues (#142816).
  • Fix a bug in associative_scan (#143048).
  • Fix the Index Put lowering with same input of self and values (#139366).
  • Fix a bug in torch.polygamma(n) when n == 0 (#144058).
  • Fix bug in integer avg_pool that was causing 0 rounding (#144059).
  • Change avg_pool with uint to match eager (#144313).
  • Fix bug in max-autotune on smaller GPUs (<68 SMs) (#145133).
  • Fix bug in torch.logit decomposition (#145576).
  • Fix bug in the strides when lowering custom op (#148367).
  • Update triton support to account for changes in AttrsDescriptor (#145051) (#145348) (#145575) (#145583) (#145515).
  • Fix bug where the benchmark_harness isn't generated, but is called in some cases (#145532).
  • Make sure not using cpp wrapper when setting nvtx training annotation (#145538).
  • Fix bug where SVE256 features were run on SVE128 systems (#146207).
  • Fix an unaligned memory access issue in mm_template (#146293).
  • Fix intermediate debug information with cpp_wrapper (#145527).
  • Fix bug where inductor was codegen-ing wrong shapes for bucketize when it was fused as an epilogue (#148769).
  • Fix bug in AOTI one-pass codegen when max-autotune is turned on (#143098).
  • Fix a memory leak in package AOTIModelPackageLoaderPybind::boxed_run (#146100).
  • Fix None and equal_to_1 arguments issue in Triton kernel generated by AOTI (#148102)
  • Fix backwards compatibility for AOTIModelPackageLoader() constructor defaults (#149082)
  • Fix blank space break windows file path (#149388)
  • Fix inductor windows linker error (#150256)
torch.fx
  • Fix get_source_partitions when weights are tied (#142446)
  • Prevent DCE of ATen rng nodes (#144319)
  • Fix incorrect type comparison (#145449)
  • Fix DCE of setitem node (#145714)
  • Fix pytree.register_constant to be usable in export (#147533)
  • Fix edge case in translation validation bisector (#145414)
torch.export
serialization
  • Rewrite the export schema format to archive without BC-breakage (#142511)
  • Serialize all dataclass fields, including default-valued members, in export schema (#142286)
  • Fix SymBool incorrectly serialized as bools (#144295)
  • Fix serialization roundtrippability for nodes with default arguments (#144686)
  • Fix deserializing bool graph outputs (#144791)
  • Fix deserialization for and_ operator (#145506)
  • Explicitly serialize unbacked_bindings (#144894)
  • Relax serialization assertion to warning for unbacked_bindings keys (#145777)
  • Avoid always printing GraphModule in de/serialization logging (#145857)
  • Bump ShapeEnv unbacked symbol counters for unbacked_bindings in deserialization (#145882)
  • Fix serialization for nested terms in nn_module_stack (#145901)
  • Fix typo in SymFloat serialization (#146112)
  • Fix deserialization for .requires_grad field (#146351)
  • Support math.trunc ops for serialization (#146715)
  • Serialize math.inf and NaN as strings (#146490)
  • Loosen SymInt input serialization for Inductor (#147237)
draft export
  • Fix dense-in-memory check for fake-kernel inference, for draft export (#145653)
  • Fix lazy_trace_handler bug in draft export logging (#146106)
  • Only clear pending unbacked symbols for overwritten fake-kernels for draft export (#147427)
  • Ignore when real-tensor fallback fails in draft export (#147779)
miscellaneous
  • Fix dynamic shape constraint checking when non-strict retracing (#143442)
  • Fix ._modules corner case for nn_module_stack metadata in strict-mode (#142823)
  • Fix placeholder name ordering for kwargs in non-strict mode (#144278)
  • Extend support for distributed ops (all_reduce, all_gather, all_gather_into_tensor, all_to_all_single, reduce_scatter_tensor) in non-strict mode (#147133, #147417)
  • Fix error with unflattener submodule reordering (#146181)
  • Make stack_trace field optional in insert_custom_op_guards pass (#146438)
  • Differentiate ScriptModules and ScriptObjects for TorchBind (#147399)
  • Restore lost input mutations with export_tracepoint (#148709)
  • Symintify transpose_ (#149057)
ONNX
  • Support subgraphs with 1+ outputs (#145860)
  • Delete rename_dynamic_shapes_with_model_inputs (#146002)
  • Handle number of outputs in builder (#147164)
  • Fix missed None type support in dynamic_shapes string cases (#148025)

Performance

Release Engineering
  • Add perf testing on H100 (#146868, #147947)
Sparse Frontend
  • Remove unnecessary tensor clones throughout codebase (#148159)
Distributed
Distributed Checkpoint (DCP)
  • Introduce process based async checkpointing (#147039)
c10d
  • Changed ALLOC_BUFFER_SIZE from 4000 to 4096 to be a power of 2 for TCPStore (#145759)
  • Improved IPC tensor release performance by releasing the IpcMutex when deleting the ExpandableSegments object and the GIL in WorkNCCL destructor (#148805)
CPU
General
  • Simplify vec128 bfloat16/half fmadds (#144486)
  • Parallelize sort (#142391)
x86
  • Set prop_kind to forward_inference when grad is not needed for mkldnn_convolution_pointwise (#142855)
  • Support reduce ops for add and max (#144065)
  • use zero-point to decide conv src zp mask (#149473)
CUDA
  • Let PYTORCH_NO_CUDA_MEMORY_CACHING has effect only when value is 1 (#145905)
  • Fix race condition in cuda initialization (#143238)
  • Fix a few 64-bit indexing issues, account for number of threads in complex128 scan (#143401)
  • Fix acquire pattern (correctness with respect to memory model) in topk (#144945)
  • Int64 indexing fix for UpSampleNearest3D (#144865)
  • Fix printing of the number of GPUs when certain asserts are raised (#146838)
  • Update the number of threads in avg_pool2d backward for SM 10.0 to prevent runtime crash (#145669)
  • Only use f8f8bf16 rowwise scaled matmul to SM 9.0 (precedes #148421 adding of kernel) (#145728)
  • Fix 64-bit indexing for Upsample2D (#141923)
MPS
  • Faster integer batched matmul (#147877)
  • Implement linear1d as shader (#148154)
  • Metal unary kernel for sqrt (#148272)
  • Faster unary operations for strided tensors (#148350)
  • Introduce strides unary op (#148468)
  • Implemented masked_fill_scalar as shader (#147369)
  • Implement bilineard2d as shader (#145581)
  • Optimize Cholesky (#145722)
  • Speedup interpolation (#148277)
ROCm
  • Improve backwards indexing when stride is not one (#147630)
  • Improvements for vectorized elementwise kernels (#143269)
  • Skip L1 cache for single-use buffers in tl.load (#143115)
  • Improve performance of reduce sum for 3D shapes (#143137)
  • Enable _load_dwordx4 ISA for BFloat16 and Half (#141397)
  • Improve reduce sum calculation for low CU count (#141378)
  • Tune 3d tensor sums when not using fastest dimension (#146170)
  • Optimize the stride one indexing backwards kernel (#146420)
  • Use IPT=8 for block radix sort (#147657)
  • Improve performance of reduce sum for 3D shapes (#143137)
  • change preferred blas lib defaults (#150212)
XPU
  • Optimize SDPA Inference Performance for XPU (#147614, #147612)
  • Improve zero-point memory creation (#148640)
  • Avoid unnecessary copy when the destination tensor of Matmul is non-contiguous or input is broadcasted (#144759, #143784)
torch.compile
Dynamo
  • Implement dynamic shape guards in C++ (#139899)
  • Directly access Python frame locals in guard checks (#140063)
  • Misc. Dynamo tracing time improvements (#143066)
Inductor
  • Support for Arm Neon and SVE support for FP32 Gemm Wrapper (#144327).
  • New GEMM kernel: persistent_tma (#142101).
  • Enable CPP Grouped GEMM Template (#143796).
  • Auto-tuning support for i8 x i8 -> i32 GEMM kernel on AMX ISA (#143187).
  • Add new GEMM templates for CPU AVX512: _weight_int4pack_mm_for_cpu (#146756).
  • Fuse SmoothQuant int8 linear pattern (#142036).
  • Add torchao da8w8 pattern with symmetric quantized activations and weights (#142110).
  • Support tiling reduction dimensions: Instead of having a single reduction dimension called "r", we can now support 2D reductions with "r0_" and "r1_" dimensions. 2D reductions generate two nested loops, with different block pointer advancements in each loop body (#137243).
  • New config to skip L1 cache for single-use buffers in triton codegen (#143115).
  • Implement max_pool2d_with_indices as a reduction for large window sizes (#147876).
  • Optimize the heuristics of outer loop fusion in Inductor CPU backend (#147523).
  • Support parallel reduction for GroupNorm in Inductor CPU backend (#144020).
  • Add support for online softmax. Online softmax uses a customized reduction to compute max and sum at the same time by accessing the data in one pass (#127011).
  • Add ROCm specific matmul tuning parameters (#148437).
torch.fx
  • Micro-optimization in Graph.nodes.__iter__ (#144631)
  • Micro-optimization in map_aggregate(immutable_dict) (#147691)
  • Move DCE rand check to import time (#145118)
Quantization
  • Enable fast qlinear static/dynamic path for AArch64 through ACL directly (#148585)
  • Improve KleidiAI 4 bit kernel performance (#146476)
  • Add NEON implementation for 8 bit quantized embedding bag on AArch64 to improve performance by ~5.5x on Neoverse V1 cores (#147322)

Documentation

Python Frontend
  • Fix description of input in torch.addbmm() (#146664)
  • fix numpy docs reference (#147697)
  • Add torch.cat type promotion documentation (#141339)
  • Add details torch.topk indices stability when duplicate values (#143736)
  • Add overloads to torch.diagonal documentation (#144214)
  • remove incorrect warnings from torch.{min,max} documentation (#146725)
  • Update addbmm, addmm, addmv and baddbmm description (#146689)
  • Fix torch.max optional args dim, keepdim description (#147177)
  • Update torch.bucketize documentaion (#148400)
  • Fix docs recommending inefficient tensor op order (#144270)
Autograd
  • Suppress vmap warning from torch.autograd.gradcheck #144287
Nested Tensor (NJT)
  • Update OSS nested tensor docs to focus on NJT (#145402)
torch.nn
  • Add clarification for target types in CrossEntropyLoss doc (#145444)
torch.optim
  • Clarify what we mean by decoupled weight decay in the *AdamWs (#144101, #144984)
  • Corrected description of AMSGrad algorithm (#142351)
Build Frontend
  • Removing doc references to PRE_CXX11_ABI. (#149756)
Distributed
FullyShardedDataParallel2 (FSDP2)
  • Highlighted equivalence of set_requires_gradient_sync and no_sync (#148715)
Distributed (c10d)
  • Updated docs for wait() (#143305)
  • Added comments to the end of Macro for better readability (#144789)
DTensor
  • Added some documentation for from_group API and add a 2D test (#146364)
  • Expose the __create_chunk_list__ in the doc (#144100)
DistributedStateDict (DSD)
  • Updated the document to mention the limitation of set_optimizer_state_dict (#148918)
Torch Elastic
  • Replaced incorrect .. note:: invocations (#142868)
  • Fixed the doc string for record (#146968)
Pipelining
  • Updated tutorials and documentation (#143045)
CUDA
  • Correct docs for clock_rate to MHz, fixes #147098 (#147393)
XPU
  • Improve "Getting Started on Intel GPU" hardware requirements and notes(#147802, #148168, #150397)
  • Improve SYCL extension, source build and AOT Inductor documentation (#147988, #143476, #149299)
  • Update Doc for Intel XPU Profiling (#134515)
  • Update CMAKE_PREFIX_PATH for XPU windows README (#148863)
torch.compile
Dynamo
  • Remove the suggestion to use suppress_errors on compiler error (#146553)
  • Automatically generated Dynamo docs (#146736)
Inductor
  • Spruce up docs for emulate_precision_casts (#145579).
  • Minor fixes to export and AOTI docs (#144513).
  • Update AOTI tutorial (#143390).
  • inductor.config.descriptive_names = False is no longer a suggested option (#145523).
torch.fx
  • Improve logging for splitter (#143771)
  • Update literal typing for torch/fx/graph nodelist (#144650)
  • Improve typing for torch/fx/_pytree.py and torch/utils/_pytree.py (#145173)
  • Fix minor mistake in docstring of replace_pattern (#147611)
torch.export
  • Export Programming Model: #143546
  • Update dynamic shapes docs for dims() and suggested fixes parser: #142510
  • Clean up docstring for torch.export.load(): #141490
Quantization
  • Add torchao docs link to PyTorch libraries (#145412)
ONNX
  • Update TorchDynamo-based ONNX Exporter memory usage example code. (#144139)
  • Deprecation message follow up (#147005)
  • Expose verification utilities (#148603)

Developers

Python Frontend
  • Collect packages with importlib in collect_env (#144616)
  • added __add__ and __mul__ hints to torch.Size (#144322)
Distributed
FullyShardedDataParallel2 (FSDP2)
  • Enabled the typing of fully_shard so that the return value can be chained with typing enabled (#147489)
Distributed (c10d)
  • Improved the dump mechanism for flight recorder (#143446)
  • Added log trace capture enabled or not in flight recorder (#143865)
  • Added file flush in file based dumper of flight recorder (#145458)
  • Caught c10 error and log message inside monitoring thread (#145413)
  • Added an API to get the status/error code at the PG level (#144498)
  • Moved record param for init to the right place (#148571)
  • Enabled testing generelization for multiple accelerator devices (#139749)
TensorParallel
  • Added warning when module is distributed twice (#147006)
Pipelining
  • Improved shape inference debug logging (#144929)
MPS
  • Support includes in metal objects (#145087)
  • Context manager to capture Metal commands for debugging/profiling (#144561)
XPU
  • Reduce the binary size of the XPU Windows package (#148313)
  • Add Python 3.13 build for XPU (#146614)
  • Make XPU Triton build supports manylinux 2.28 (#148195)
  • Fix XPU builds inside venv (#150300)
Benchmark
  • Remove old ONNX benchmarks from operator benchmarks (#146325)
  • Add option to write operator benchmark output to a JSON (#142809)
  • Improve operator benchmark results parsing (#144297)
  • Add more operators {add_, addcmul, arange, baddbmm, bmm, clamp, div, div_, gelu, index_add, logical_and, mul_, sub_, topk, where} to operator benchmark (#145625)
  • Add cachebench to operator benchmarks for PT2 caching (#147537)
torch.compile
Dynamo
  • New internal graph break API that enforces better error messages (#146525)
  • Replace internal calls to torch._dynamo.optimize() with torch.compile() (#142451)
Inductor
  • Support for export to unwrap/wrap subclasses AOT, resolves UX issue in torchao where users had to manually unwrap their subclasses before calling export (#141941).
  • Autotuning logs will now show up in TORCH_LOGs under the name "autotuning" (#147222).
  • Replace set by OrderedSet: only use OrderedSet in the Inductor codebase (#138466).
  • Now MPS is considered a GPU_TYPE (#143634).
  • Separate unary post op fusion and lowering for qlinear (#143903).
  • New classes to help with kernel memory analysis in heuristics (#142026).
  • Move ir_pre_fusion.txt and ir_post_fusion.txt from TORCH_COMPILE_DEBUG to TORCH_LOGS. For example, TORCH_LOGS="+ir_pre_fusion" (#147248).
  • Implement deepcopy for AOTICompiledModel (#145423)
torch.fx
  • Downgrade some logs (#147538, #145075)
  • Refactor immutable collections implementation (#144640)
  • Make fx.node.map_arg() and .map_aggregate() generic (#146248)
View originalPermalink

Discussion