Keras v3.14.0

v3.14.0
Added 12
  • Added full support for Orbax checkpoints, including sharding, remote paths, and step recovery
  • Added support for Activation-aware Weight Quantization (AWQ) and Asymmetric INT4 Sub-Channel Quantization
  • Added batch renormalization feature to the BatchRenormalization layer
  • Added ScheduleFreeAdamW optimizer
  • Introduced optional Gated Attention support in MultiHeadAttention and GroupedQueryAttention layers
  • Added NaN-aware NumPy operations: nanmin, nanmax, nanmean, nanmedian, nanvar, nanstd, nanprod, nanargmin, nanargmax, and nanquantile in keras.ops.numpy
Changed 3
  • PyTorch backend now supports dynamic shapes in export and includes device selection improvements
  • JAX backend improved RNG handling in FlaxLayer and JaxLayer, variable jitting improvements, and direct JAX-to-ONNX export
  • NumPy backend now supports masking
Fixed 5
  • Fixed multiple symbolic shape bugs across layers like Conv1DTranspose, IndexLookup, and TextVectorization
  • Fixed activity regularizer normalization by batch size
  • Fixed CuDNN-based LSTM and GRU implementation in PyTorch backend
  • Improved Sequential error messages for incompatible layers
  • Minimized memory usage issues in sparse_categorical_crossentropy
Highlights
  • Orbax Checkpoint Integration: Full support for Orbax checkpoints, including sharding, remote paths, and step recovery.
  • Quantization Upgrades: Added support for Activation-aware Weight Quantization (AWQ) and Asymmetric INT4 Sub-Channel Quantization.
  • Batch Renormalization in BatchNorm: Added batch renormalization feature to the BatchRenormalization layer.
  • New Optimizer: Added ScheduleFreeAdamW optimizer.
  • Gated Attention: Introduced optional Gated Attention support in MultiHeadAttention and GroupedQueryAttention layers.
New Features and Operations
Multi-Backend Operations
  • NaN-aware NumPy Operations: Added support for nanmin, nanmax, nanmean, nanmedian, nanvar, nanstd, nanprod, nanargmin, nanargmax, and nanquantile in keras.ops.numpy.
  • New Math & Linear Algebra Operators: Added nextafter, ptp, view, sinc, fmod, i0, fliplr, flipud, rad2deg, geomspace, depth_to_space, space_to_depth, and fold.
Preprocessing and Layers
  • CLAHE Layer: Added Contrast Limited Adaptive Histogram Equalization preprocessing layer.
  • Adapt Support for Iterables: Preprocessing layers now support Python iterables in the adapt() method, which allows the direct use of Grain datasets.
OpenVINO Backend Support

The OpenVINO backend received a massive update, implementing a wide array of NumPy and Neural Network operations to achieve feature parity with other backends:

  • NumPy Operations: vander, trapezoid, corrcoef, correlate, flip, diagonal, cbrt, hypot, trace, kron, argpartition, logaddexp2, ldexp, select, round, vstack, hsplit, vsplit, tile, nansum, tensordot, exp2, trunc, gcd, unravel_index, inner, cumprod, searchsorted, hanning, diagflat, norm, histogram, lcm, allclose, real, imag, isreal, kaiser, shuffle, einsum, quantile, conj, randint, in_top_k, signbit, gamma, heaviside, var, std, inv, solve, cholesky_inverse, fft, fft2, ifft2, rfft, irfft, stft, istft, scatter, binomial, unfold, QR decomposition, view, and more.
  • Neural Network Operations: Added support for separable_conv, conv_transpose, adaptive_average_pool, adaptive_max_pool, RNN, LSTM, and GRU.
  • Control Flow Operations: Implemented cond, scan, associative_scan, map, switch, fori_loop, and vectorized_map.
Bug Fixes and Improvements
Backend Specific Improvements
  • PyTorch: Dynamic shapes support in export, device selection improvements, and bug fixes to the CuDNN based LSTM and GRU implementation.
  • JAX: Improved RNG handling in FlaxLayer and JaxLayer, variable jitting improvements, and direct JAX-to-ONNX export.
  • NumPy: Enabled masking support for the NumPy backend.
Other Improvements
  • Fixed multiple symbolic shape bugs across layers like Conv1DTranspose, IndexLookup, and TextVectorization.
  • Fixed activity regularizer normalization by batch size.
  • Improved Sequential error messages for incompatible layers.
  • Minimized memory usage issues in sparse_categorical_crossentropy.
New Contributors

We would like to thank our new contributors for making their first contribution to the Keras project:

Full Changelog: https://github.com/keras-team/keras/compare/v3.13.2...v3.14.0

View original

Upgraded? How did it go?

Discussion