# Ray: what changed from 1 to 2 - Product: Ray (https://whatsnew.fyi/product/ray) - Vendor: Anyscale - Range: changelog entries numbered after 1.13.0 up to and including 2.58.0, stable releases only - Entries below: 71 releases (newest first) - Resolved: 1 is 1.13.0 and 2 is 2.58.0, the newest stable release of each major we track - Carrying security changes: 4 · CVEs mentioned: 10 · Mentioning breaking changes: 4 · Removing or deprecating something: 27 - Page: https://whatsnew.fyi/product/ray/compare/1...2 What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog. Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'. ## What changed (1059 changes, grouped by kind) ### Added #### 2.58.0 (2026-08-23) - Add Dataset.with_columns for multi-column expression projection - Add write_delta for Delta Lake with catalog support - Add Torch inference API - Promote hash shuffle v2 to a selectable shuffle strategy with aggregation support, vectorized aggregation, and block splitting during aggregation - Add ignore_missing_paths and skip_paths to read_parquet on DatasourceV2 - Add delta_timestamps (temporal windows) to read_lerobot - Collect cluster usage metrics by sampling in background threads during execution - Tolerate actor deaths during init via DataContext.max_consecutive_actor_init_deaths - Expose RAY_DATA_HASH_SHUFFLE_MAP_TASK_TARGET_INPUT_BYTES in DataContext - Add configurable status code and Retry-After header for backpressure rejections - Add dependency-ordered shutdown for deployments - Add an optional tie-break key to best-fit node scheduling - Scale ingress request router replicas per proxy node - Ray Train integrates with TorchTPU backend - Add a public preemption API and a controller PreemptingState - Add data ingest metrics to the Train dashboard - Add experimental Ray Sandbox which runs task and actor code under gVisor and can run Docker-built images directly - Add SubslicePlacementGroup for gang scheduling on TPU subslices and single-host TPU support in SlicePlacementGroup #### 2.57.0 (2026-08-11) - Enable DataSourceV2 by default via DataContext.use_datasource_v2 - Add task-based hash shuffle v2 with ShuffleMapOp, ShuffleReduceOp, join support, multi-input reduce, downstream map fusion, and reducer remote args - Add Catalog abstraction with UnityCatalog implementation for read operations and Unity Catalog write support for Parquet and Iceberg - Add read_zarr for Zarr datasets - Add read_lerobot for LeRobot v3 datasets - Add PushdownCountFiles optimization to answer count() from Parquet footers - Add common subexpression elimination to the expression optimizer - Add GPU support for Aggregate - Make dataset iteration metrics queryable per split - Add custom operator stats to capture worker-side metrics during task execution - Add embedded RocksDB storage backend for GCS fault tolerance selectable with RAY_gcs_storage=rocksdb and RAY_gcs_storage_path - Add public API for topology-aware scheduling - Add gRPC support to HAProxy direct-ingress path including streaming, metrics, and custom request IDs - Add experimental KV-cache-aware request routing that tracks replica KV state, tokenizes before routing, and routes on prefill/decode token load #### 2.56.0 (2026-06-29) - Support multiple datasets per cluster via subcluster labels and resource partitioning - Add Dataset.mix() public API and MixOperator for weighted dataset mixing - Implement new DataSourceV2 framework with ParquetDatasourceV2, chunked reader, predicate splitting, and listing/scanner infrastructure - Add batch_size='auto' to map_batches to derive batch row count from target row batch size - Implement distributed upsert for Iceberg using task-based merge algorithm - Add include_row_hash to read_parquet - Add JAX data iterator - Expose isolate_read_workers flag to run read tasks on isolated worker processes - Expose default_map_logical_memory_enabled flag to set default logical memory for map operators - Support predicate pushdown for Lance format - Support per-partition start_offset and end_offset for read_kafka - Add obstore async download backend for download operator - Support UDF retries on transient exceptions - Add GPU-domain-aware placement groups using label locality for packing bundles onto nodes with ray.io/gpu-domain label - Add initial Kubernetes in-place pod resizing support for Autoscaler v2 - Introduce session-sticky routing via consistent hashing with ConsistentHashRouter - Add CapacityQueueRouter for supply-constrained workloads #### 2.55.0 (2026-04-15) - Add DataSourceV2 API with scanner/reader framework, file listing, and file partitioning - Support GPU shuffle with rapidsmpf 26.2 - Add Kafka datasink, migrate to confluent-kafka, support datetime offsets - Add Turbopuffer datasink - Add 2-phase commit checkpointing with trie recovery and load method - Integrate queue-based autoscaling policy with task consumers - Enable autoscaling for GPU stages - Add random(), uuid(), cast, and map namespace support to Expressions - Add support for Arrow native fixed-shape tensor type - Support writing tensors to tfrecords - Add pathlib.Path support to read_* functions - Add cudf as a batch_format - Allow ActorPoolStrategy for read_datasource() via compute parameter - Introduce ExecutionCache for streamlined caching - Support strict=False mode for StreamingRepartition - Add monotonically increasing ID support #### 2.54.0 (2026-02-18) - Add checkpointing support to Ray Data - Add list operations, fixed-size arrays, string padding, logarithmic, trigonometric, arithmetic, and rounding to Compute Expressions - Add sql_params support to read_sql - Add AsList aggregation - Support CountDistinct aggregate - Add credential provider abstraction for Databricks UC datasource - Support callable classes for UDFExpr - Add autoscaler metrics to Data Dashboard - Add optional filesystem parameter to download expression - Allow specifying partitioning style or flavor in write_parquet() - Enable new cluster autoscaler by default - Add Queue-based autoscaling for TaskConsumer deployments with QueueMonitor actor support for Redis and RabbitMQ #### 2.53.0 (2025-12-20) - Add Kafka as a native datasource for data ingestion - Add Dataset.summary() API for quick dataset inspection - Add Iceberg schema evolution, upsert, and overwrite support - Add should_continue_on_error for graceful error handling in batch inference - Add datetime compute expressions support - Enable expressions for grouped with_column in Ray Data - Add new utilization-based cluster autoscaler for Ray Data workloads - Add deployment topology visibility in Serve REST API to visualize deployment dependency graphs - Add external_scaler_enabled flag to application config for third-party autoscaler integration - Add node rank and local rank support to replica rank system for multi-node deployments - Add custom batch size function support in @serve.batch - Add policy state persistence for custom autoscaling policies - Add Prometheus metrics for autoscaling decisions and router queue latency #### 2.52.0 (2025-11-21) - Added predicate pushdown rule that pushes filter predicates past eligible operators - Iceberg support for upsert tables, schema updates, and overwrite operations - Iceberg support for predicate and projection pushdown - Iceberg write datafiles in write() then commit - Namespaced expressions that expose PyArrow functions - Added version argument to read_delta_lake - Generator UDF support for map_groups - ApproximateTopK aggregator - Serialization framework for preprocessors - Support for temporal, list, tensor, and struct datatypes #### 2.51.0 (2025-10-29) - Application-level autoscaling in Ray Serve with custom autoscaling policies that operate across all deployments in an application - Enhanced autoscaling context with replica-level metrics including total_running_requests, total_queued_requests, and total_num_requests - Autoscaling metrics aggregation functions supporting min, max, and time-weighted average aggregation - Support for multiple task consumers in a single Ray Serve application - Enhanced support for Unity Catalog integration in Ray Data - New expression evaluator infrastructure for improved query optimization in Ray Data - Support for SaveMode in Ray Data write operations - Approximate quantile aggregator in Ray Data - MCAP datasource support for robotics data in Ray Data - Callback-based stat computation for preprocessors and ValueCounter in Ray Data - Support for multiple download URIs with improved error handling in Ray Data - Top-level ray.train aliases for public APIs #### 2.50.0 (2025-10-10) - New expression API enables predicate-based filtering, UDF transformations with with_column, and column aliasing for more powerful data transformations - Added support for projection pushdown into Parquet reads - New download expression enables efficient loading of data from columns containing URIs with improved performance and error handling - New explain() API provides insights into dataset execution plans - Added streaming_train_test_split to avoid materialization for train/test splits - Enabled multi-node tensor and pipeline parallelism for Ray Data LLM processing - Added chat_template_kwargs parameter for customizing chat templates in Ray Data LLM - Added support for OpenAI's nested image URL format in Ray Data LLM multimodal pipelines - vLLM engines can now be shared across sequential processors for better resource utilization - Enhanced Dataset.stats() output with input/output row counts per operator - Added new metrics for task duration, inputs per task, and output blocks - Added time to first batch metric for better iteration performance monitoring - Added type-specific aggregators for numerical, categorical, and vector columns - Added fine-grained concurrency controls with max_task_concurrency and resource allocation options - Alpha release of Ray Direct Transport enabling GPU data to remain in GPU memory with efficient data transfer via collective communication and RDMA - Added StepFailedRecreateEnv exception for users with unsatisfiable environments in RLLib - Async Inference alpha release for Ray Serve with APIs supporting long-running asynchronous inference tasks and message broker support - Added support for replica ranks in Ray Serve for large-model inference use-cases - Added FastAPI factory pattern support in Ray Serve for using non-serializable FastAPI plugins - zip() operator now supports combining multiple datasets, not just pairs #### 2.49.0 (2025-08-26) - Wrapped batch indices in a BatchMetadata object to make per-batch metadata explicit - Added support for Anti/Semi Join types - Introduced an Issue Detection Framework - Added an option to enable out-of-order execution for better performance - Introduced a StreamingSplit logical operator for DAG rewrite - Added a stratify parameter to train_test_split - Added Snowflake connectors - Updated Hudi integration to support incremental query - Added an Actor location tracker - Added BundleQueue.has_next - Made DEFAULT_OBJECT_STORE_MEMORY_LIMIT_FRACTION configurable - Added Expression support and a with_columns API - Allocate GPU resources in ResourceManager - Introduced JaxTrainer with SPMD support on TPUs #### 2.48.0 (2025-07-18) - Add reading from Delta Lake tables and Unity Catalog integration - Enhanced pin_memory support in iter_torch_batches - Added broadcast_from_rank_zero and barrier collective operations - Enhanced hardware metrics integration with Grafana dashboards - Added support for dynamically loading callbacks via environment variables - Added RouterConfig field to DeploymentConfig for custom RequestRouter configuration - Added support for implementing custom request routing algorithms - Support separate deployment config for PDProxy in PrefixAwareReplicaSet - Support for prefix-aware request router #### 2.47.0 (2025-06-12) - Support prefill disaggregation in Ray Serve LLM for production LLM serving use cases - Add save modes support to file data sinks - Add flattening capability to the Concatenator preprocessor to support output vectorization use cases - Support for configuring Ray Train worker actor runtime environments - Include Grafana panel data in Ray Train export for improved monitoring - Introduce a structured logging environment variable to standardize log formats - Add metrics for TrainControllerState to enhance observability - Add support for implementing custom request routing algorithms in Ray Serve - Introduce an environment variable to prioritize custom resources during deployment scheduling - Expose vLLM Metrics to serve.llm API - Add Embedding API to Ray Serve/Data LLM - Add GPU inference to offline evaluation in RLlib #### 2.46.0 (2025-05-07) - Ray Data now supports hash-shuffle based repartitioning and aggregations - Ray Data now supports joins using hash-shuffle - Ray Serve LLM is now compatible with vLLM v1 - Add initial Train Grafana dashboard with built-in metrics for training workloads - Offline Evaluation with loss function for Offline RL pipeline, introducing three new callbacks: on_offline_evaluate_start, on_offline_evaluate_end, on_offline_eval_runners_recreated - New custom_data attribute for SingleAgentEpisode and MultiAgentEpisode to store custom metrics #### 2.45.0 (2025-04-29) - Support with_tensor_transport(transport='shm') in cgraph - Support reduce scatter and all gather collective for GPU communicator in compiled graph - Add ClickHouse sink via Dataset.write_clickhouse() - Support ray_remote_args_fn in Dataset.groupby().map_groups() to set per-group runtime env and resource hints - Expose Dataset.name and set_name as public API for easier lineage tracking - Allow async callable classes in Dataset.flat_map() - Introduce Ruleset abstraction for rule-based query optimisation - Add seamless conversion from Daft DataFrame to Ray Dataset - Improve support for line-delimited JSONL reading in read_json() - Provide Dataset.export_metadata() for schema and stats snapshots #### 2.44.0 (2025-03-21) - Ray Compiled Graph (beta) with less than 50us system overhead for repeatedly executed task graphs and native GPU-GPU communication via NCCL - Iceberg write support through pyiceberg in Ray Data - LoRA support and structured outputs enhancements to Ray Data LLM - State export API in Ray Train - Faster bulk imperative Serve Application deploys - Gen-config for Ray Serve LLM - Dataset/operator state, progress, and total metrics in Ray Data #### 2.43.0 (2025-02-27) - New module ray.data.llm for batch inference with large language models, featuring vLLMEngineProcessorConfig and HttpRequestProcessorConfig abstractions (alpha) - New module ray.serve.llm for serving large language models - Ray Train V2 available for testing with RAY_TRAIN_V2_ENABLED=1 environment variable - Integration with uv run for specifying Python dependencies for Ray cluster drivers and workers - Support class constructor args for filter() #### 2.42.0 (2025-02-05) - Add read_audio and read_video for Ray Data - Add Train v2 user-facing callback interface in Ray Train - Add TuneReportCallback for propagating intermediate Train results to Tune - Add metrics to replay buffers in RLlib #### 2.41.0 (2025-01-23) - Expression support for filters in Ray Data - Support partition_cols in write_parquet for Ray Data - Multi-directional sort over Ray Data datasets - Output JSON structured log files for system and application logs in Ray Train - Support for AMD ROCR_VISIBLE_DEVICES in Ray Train - Allow users to configure optuna storage in optuna_search - Support for external Envs to new API stack in RLlib with custom tcp-capable EnvRunner - Sequence sampling to EpisodeReplayBuffer in offline RL - ExecutionCallback interface in Ray Data - Seed for read files in Ray Data - Support for individual lambda-style callbacks in RLlib Callbacks API #### 2.40.0 (2024-12-04) - Add read_hudi function to Ray Data - Add initial version of local_testing_mode to Ray Serve - Add uv runtime env support to Ray Core #### 2.39.0 (2024-11-13) - Cancelled requests now return a serve.RequestCancelledError - Exposed application source in app details model - Add off-policy'ness metric to new API stack - Validate episodes before adding them to the buffer - Support all reduce collective in aDAG - Add visualization of compiled graphs - Support asyncio.gather on multiple CompiledDAGFutures #### 2.38.0 (2024-10-23) - Add Dataset.rename_columns - Add basic structured logging to Ray Data - Add partitioning parameter to read_parquet - Enable cloud checkpointing in RLlib - CompiledGraphs: support multi readers in multi node when DAG is created from an actor #### 2.37.0 (2024-09-24) - Allow control of some serve configuration via env vars - Add restart-failed-env option to EnvRunners - Add episode sampling to EpisodeReplayBuffer - Add SelfSupervisedLossAPI for RLModules that bring their own loss - Add InferenceOnlyAPI - Allow custom NCCL group for aDAG - Support buffered input for aDAG - Support multi node multi reader for aDAG #### 2.36.0 (2024-09-17) - Allow user-defined Exception to be caught in Ray Data - Decouple device-related modules and add Huawei NPU support to Ray Train - Setup default serve logger - Add CQL algorithm to new API stack in RLlib - Enable GPU and multi-GPU support for DQN/SAC/CQL in new API stack - Add torch.compile config options to old API stack - Add kwargs to torch.nn.parallel.DistributedDataParallel - Add PREPARED state for placement groups in GCS for better fault tolerance #### 2.35.0 (2024-08-28) - Add support for reading from Iceberg in Ray Data - Add failure retry logic for read_lance in Ray Data - Add hardware stats in Ray Train #### 2.34.0 (2024-07-31) - Add better support for UDF returns from list of datetime objects - Initial design for Ray-Data based offline RL Algos on new API stack - Add user-defined schemas for data loading - Make data pipeline better configurable and tuneable for users #### 2.33.0 (2024-07-25) - Add read API for reading Databricks table with Delta Sharing - Add support for objects to Arrow blocks #### 2.32.0 (2024-07-10) - aDAG Developer Preview: Ray accelerated DAGs with Ray Core-like API and extensibility to pre-compile execution paths across pre-allocated resources - Support async callable classes in map_batches() - APPO on new API stack with EnvRunners - Added customizable refresh frequency for metrics on Ray Dashboard #### 2.31.0 (2024-06-26) - Added documentation for dataset.Schema - Add API for Ray Train run stats - IMPALA runs on the new API stack with EnvRunners and ConnectorV2s - SAC/DQN: Prioritized multi-agent episode replay buffer #### 2.30.0 (2024-06-20) - Create internal request id to track request objects - Alpha release of job level logging configuration allowing users to configure user logging in logfmt format with logging context #### 2.24.0 (2024-06-06) - Allow user to configure timeout for actor pool - Add override_num_blocks to from_pandas and perform auto-partition - Add support for multi-agent off-policy algorithms (DQN and SAC) in the new API - Config option for APPO/IMPALA to change number of GPU-loader threads - Ray c++ backend structured logging #### 2.23.0 (2024-05-22) - Add support for using GPUs with map_groups - Add support for using actors with map_groups #### 2.22.0 (2024-05-14) - Add function to dynamically generate ray_remote_args for Map APIs - Allow manually setting resource limits for training jobs #### 2.21.0 (2024-05-08) - Add read_lance API to read Lance Dataset - New MetricsLogger API now fully functional on the new API stack, working inside Learner classes and loss functions #### 2.20.0 (2024-05-01) - Track Train Run Info with TrainStateActor - Introduce MetricsLogger, a unified API for logging custom metrics and stats in all of RLlib's components - Introduce inference-only (slim) mode for RLModules that run inside an EnvRunner - Add MultiAgentEpisodeReplayBuffer for new API stack in preparation for multi-agent support #### 2.12.0 (2024-04-25) - Store Ray Data logs in special subdirectory - Add in `local_read` option to `from_torch` - Support of images and video logging in WandB - Example script to pre-train an `RLModule` in single-agent fashion, then bring checkpoint into multi-agent setup and continue training #### 2.11.0 (2024-04-17) - Support reading Avro files with ray.data.read_avro - Add experimental support for AWS Trainium (Neuron) - Add experimental support for Intel HPU #### 2.10.0 (2024-03-21) - Ray Data becomes generally available with stability improvements in streaming execution, reading and writing data, better tasks concurrency control, and debuggability improvement with dashboard, logging and metrics visualization - New API Stack officially announced as alpha for PPO and SAC in RLlib - Added default autoscaling policy set via num_replicas="auto" in Serve - Added support for active load shedding via max_queued_requests in Serve - Added replica queue length caching to the DeploymentHandle scheduler in Serve - Autoscaler v2 is in alpha for Kuberay with improved observability and stability - Added support for accelerator types via ScalingConfig(accelerator_type) in Train - Add num_rows_per_file parameter to file-based writes in Ray Data - Add DataIterator.materialize in Ray Data - Skip schema call in DataIterator.to_tf if tf.TypeSpec is provided in Ray Data - Add option to append for Dataset.write_bigquery in Ray Data - Add backend-specific context manager for train_func for setup/teardown logic in Train #### 2.9.1 (2024-01-19) - Add debugpy as the ray debugger - Add exponential backoff for queue_len_response_deadline_s #### 2.9.0 (2023-12-21) - Add Ray Dashboard page for Ray Data to monitor real-time execution metrics - Make streaming generator officially a public API for writing streaming applications on top of Ray via Python generator API - Add experimental support for Intel GPU accelerators - Add experimental support for Intel Gaudi Accelerators - Add experimental support for Huawei Ascend NPU accelerators - Add concurrency argument to Ray Data map-like APIs to replace ComputeStrategy - Allow task failures during Ray Data execution - Add new API for reading and writing Datasource in Ray Data - Enable group-by over multiple keys in Ray Data datasets - Add support for multiple group keys in map_groups for Ray Data - Introduce logging config in Ray Serve to set different logging parameters for different applications and deployments - Add gRPC context object into gRPC deployments in Ray Serve for users to set custom code and details back to the client - Introduce experimental runtime environment feature in Ray Serve to run applications in different containers with different images - Support reading Result from cloud storage in Ray Train and Ray Tune #### 2.8.0 (2023-11-03) - Add support for shuffling input files in Ray Data - Support streaming read of PyTorch dataset - Add BigQuery datasource for Ray Data - Add Databricks table and SQL datasource for Ray Data - Add inverse transform functionality to LabelEncoder - Add function arg params to Dataset.map and Dataset.flat_map - Add initial support for scheduling workers on neuron_cores in Ray Train - Add new Java APIs that aligns with the Ray Serve 2.x APIs - Add new metrics for Ray Data in the Metrics tab on the Ray Dashboard - Ray Core now supports profiling GPU tasks or actors using Nvidia Nsight #### 2.7.1 (2023-10-09) - Add an `application` tag to the `ray_serve_num_http_requests` metric - Add a new user guide for the vSphere cluster launcher #### 2.7.0 (2023-09-17) - Ray Serve and Ray Train PyTorch support are now generally available - Introduce new backwards-compatible DeploymentHandle API in Ray Serve to unify various existing Handle APIs - Add high performant gRPC proxy to Ray Serve to serve gRPC requests - Consolidate various PyTorch-based trainers into TorchTrainer in Ray Train - Introduce new train.Checkpoint API providing consolidated way of interacting with remote and local storage - Add initial integrations with TPUs and AWS accelerators in Ray Core to natively detect these devices and schedule tasks/actors onto them - Add official support for actor task cancellation in Ray Core - Add experimental streaming generator in Ray Core that supports streaming response to the caller - Integrate Ray Core streaming generator API by default in Ray Data to reduce memory footprint throughout the data pipeline - Add Dataset.write_images to write images - Add Dataset.write_sql() to write SQL databases - Support sort on multiple keys in Ray Data - Support reading and writing JSONL file format in Ray Data - Support class constructor args for Dataset.map() and flat_map() - Implement streamed read from Hugging Face Dataset #### 2.6.3 (2023-08-15) - Add Learner API documentation for RLlib #### 2.6.0 (2023-07-21) - Support for HTTP streaming response and WebSockets is now enabled by default in Ray Serve - Ray Serve @serve.batch-decorated methods can now stream responses - Support for restoring Results from local trial directories - New streaming integration of Ray Data and Ray Train allowing streaming data ingestion for model training and per-epoch data preprocessing - Add Dataset.unique() method - Add option for parallelizing post-collation data batch operations in DataIterator.iter_batches() - LightningTrainer support for DeepSpeedStrategy #### 2.5.1 (2023-06-21) - Build Python wheels on Mac OS for Python 3.11 #### 2.5.0 (2023-06-08) - Support for checkpointing distributed models in Ray Train - PyTorch Lightning FSDP support in LightningTrainer to enable training large models on Ray Train - Support for streaming responses and model multiplexing in Ray Serve and Core for LLM applications - Support for multi-GPU training in RLlib - Enable lightweight resource broadcasting feature in Ray Core to improve reliability and scalability - Add column API to Dataset - Configure progress bars via DataContext - Support using concurrent actors for ActorPool - Add take_batch API for collecting data in the same format as iter_batches and map_batches - Add alias parameters to the aggregate function and add quantile function #### 2.4.0 (2023-04-25) - Introduce LightningTrainer to scale PyTorch Lightning on Ray - Release AccelerateTrainer to run HuggingFace Accelerate and DeepSpeed on Ray - Add examples for GPT-J fine-tuning with Microsoft DeepSpeed and Ray Train - Add examples for GPT-J-6B batch prediction with Ray Data - Add examples for GPT-J-6B serving with Ray Serve - Add examples for Stable Diffusion Dreambooth fine-tuning with Ray Train - Add examples for Stable Diffusion batch prediction with Ray Data - Add examples for Stable Diffusion serving with Ray Serve - Enable streaming execution by default for more efficient data processing pipelines - Implement asynchronous batch prefetching in Dataset.iter_batches - Add support for reading SQL databases with Ray Data - Add support for reading WebDataset format - Add multi-app CLI and REST API support to Ray Serve - Add TorchDetectionPredictor - Add collate_fn parameter to iter_torch_batches #### 2.3.0 (2023-02-24) - Streaming backend for Ray Datasets in Developer Preview for terabyte-scale ML inference and training workloads - New Information Architecture for Ray dashboard organized around user personas and workflows in Beta - Ray-on-Spark is now available in Preview to launch Ray clusters on Databricks and Spark clusters - Add set_preprocessor method to Checkpoint - Introduce DatasetIterator - Add per-epoch preprocessor - Add TorchVisionPreprocessor - Persist Dataset statistics automatically to log file - Add zero-copy batch API for ds.map_batches() #### 2.2.0 (2022-12-13) - Ray Jobs API is now generally available for submitting locally developed applications to a remote Ray Cluster - Add NumPy first path for Torch and TensorFlow Predictors in Ray AIR - Add select_columns() to select a subset of columns in Ray Data - Add write_tfrecords() to write TFRecord files in Ray Data - Support MongoDB data source in Ray Data - Add from_torch() to create dataset from Torch dataset in Ray Data - Add from_tf() to create dataset from TensorFlow dataset in Ray Data - Allow to set batch_size in BatchMapper in Ray Data - Support read/write from/to local node file system in Ray Data - Add MosaicTrainer in Ray Train #### 2.1.0 (2022-11-08) - Ray Datasets read_images() API for loading image data - Ray Datasets read_tfrecords() API to read TFRecord files - Numpy-based API for user-defined functions in Preprocessor - Ray Serve support for gRPC endpoint (alpha release) - RLlib decision transformer (DT) algorithm - RLlib on_episode_created() callback hook - Learning rate schedule to SimpleQ and PG algorithms - Ray OOM prevention (alpha release) - Support dynamic generators as task return values - Dashboard time series metrics support - Dashboard export configuration files for Prometheus or Grafana - Dashboard new progress bar in job detail view - NumPy batch format to Preprocessor and BatchMapper - Ragged tensor extension type - KBinsDiscretizer Preprocessor - FullyShardedDataParallel support to TorchTrainer - Tuner.get_results() to retrieve results after restore #### 2.0.1 (2022-10-22) - Allow host and port in Serve config - RLlib evaluation supports asynchronous rollout #### 2.0.0 (2022-08-23) - Ray AIR, a scalable and unified toolkit for ML applications, is now in Beta - Ray now supports natively shuffling 100TB or more of data with the Ray Datasets library - KubeRay, a toolkit for running Ray on Kubernetes, is now in Beta - Ray Serve's Deployment Graph API is a new and easier way to build, test, and deploy an inference graph of deployments - BatchPredictors now have support for scalable inference on GPUs - All Predictors can now be constructed from pre-trained models - Add ImageFolderDatasource - Add the NumPy batch format for batch mapping and batch consumption - Add iter_torch_batches() and iter_tf_batches() APIs - Add local shuffling API to iterators - Add drop_columns() API - Add randomize_block_order() API - Add random_sample() API - Add support for len(Dataset) - Add UDF passthrough args to map_batches() - Add Concatenator preprocessor ### Changed #### 2.57.0 (2026-08-11) - Hash Shuffle V2 replaces aggregator actor pool with two stateless task-based operators that pass shards through the object store - Distribute HAProxy ingress as the ray-haproxy PyPI package instead of compiling into images, making it the default HAProxy binary - Default read_numpy to allow_pickle=False and make it manually configurable - Make write_lance with mode=CREATE error instead of silently overwriting #### 2.56.1 (2026-07-17) - The threshold memory monitor now snapshots both user and system cgroup slices and logs an error when system-slice usage exceeds reserved system memory #### 2.56.0 (2026-06-29) - Fix iter_batches spilling by replacing make_async_gen with iter_threaded and reducing buffered batches - Re-architect Ray Serve LLM by decoupling request handling from token streaming response path #### 2.55.1 (2026-04-22) - Upgrade apt packages in slim base #### 2.55.0 (2026-04-15) - Cache _map_task args, improve heap-based actor ranking, and improve actor pool map - Optimize concat tables and PyArrow schema hashing - Reduce default DownstreamCapacityBackpressurePolicy threshold to 50% - Improve reproducibility for random APIs - Port changes from lance-ray into Ray Data - Enable PyArrow compute-to-expression conversion for predicate pushdown - Upgrade to pyiceberg 0.11.0 and cap pandas to <3 - Refactor logical operators to frozen dataclasses #### 2.54.0 (2026-02-18) - Improve numerical stability in scalers by handling near-zero values - Update to PyArrow 23 - Demote Sort from requiring preserve_order - Use Arrow IPC for Arrow Schema serialization and deserialization - Enable and tune DownstreamCapacityBackpressurePolicy - Decouple Ray Train from Ray Data by removing top-level ray.data imports - Move extension types to ray.data #### 2.53.0 (2025-12-20) - Parallelize DefaultCollateFn and arrow_batch_to_tensors - Optimize autoscaler to support configurable step size for actor pool scaling - Improve streaming repartition performance - Add version support to read_lance() - Add configurable batching for resolve_block_refs to speed up iter_batches #### 2.52.0 (2025-11-21) - Enhanced Unity Catalog integration - Use approximate quantile for RobustScaler preprocessor - Map batches support for limit pushdown - Make all map operations zero-copy by default - Use tqdm_ray for progress reporting from workers - Improved concurrency cap backpressure tuning #### 2.51.0 (2025-10-29) - Ray Train v2 is now enabled by default, providing usability and stability improvements and new features - Improved projection pushdown handling with renamed columns in Ray Data - Enhanced hash-shuffle performance with better retry policies in Ray Data - Reduced memory usage with improved streaming generator backpressure in Ray Data #### 2.50.0 (2025-10-10) - Default shuffle strategy changed from sort-based to hash-based for improved shuffle performance and lower peak memory usage for aggregations - Concurrency parameter now accepts tuples for more flexible configuration - Resource requirements (num_cpus/gpus, memory) are now top-level parameters in most APIs for easier configuration - Write operations now use iterators instead of accumulating blocks in memory - Improved groupby performance with sort-shuffle pull-based approach - Improved join operations with new abstractions - Improved compatibility between PyArrow native types, extension types, and pandas Arrow dtypes - Joins now supported with list/tensor non-key columns - Enhanced support for variable-shaped tensor arrays with different dimensions - Added serialization/deserialization for PyArrow Extension Arrays - Reduced memory usage for OneHotEncoder - Reduced memory usage for schema unification - Eliminated unnecessary block copying and double execution of arrow conversions - Improved Parquet encoding ratio estimation - Enabled per-block limiting for Limit operator - Optimized schema handling with deduplication and removed unnecessary unification #### 2.49.0 (2025-08-26) - Decoupled actor and node autoscaling; autoscaling now also considers budget - Faster hash-shuffle resource usage calculation and more accurate shuffle progress totals - Reduced Parquet metadata storage usage - map_batches now defaults to row_modification=False and avoids pushing past limit - ray.train.get_dataset_shard now lazily configures dataset sharding for better startup behavior #### 2.48.0 (2025-07-18) - Enhanced handling of mismatched columns and pandas.NA values - Improved read_text trailing newline semantics - Optimized backpressure handling with policy-based resource management - Enhanced write_parquet with support for both partition_by and row limits - Improved execution performance for One Hot encoding in preprocessors - Improved checkpoint population from before_init_train_context - Enhanced controller state logging and metrics - Enhanced FastAPI ingress deployment validation for multiple deployments - Optimized get_live_deployments performance #### 2.47.0 (2025-06-12) - Re-enable Actor locality-based scheduling with improved algorithms for ranking bundle locations - Disable blocking pipeline by default until Actor Pool fully scales up to min actors - Improve progress bar and dashboard to show name of partial functions properly - The ingress API now accepts a builder function in addition to an ASGI app object - Allow setting name_prefix in build_llm_deployment - Mitigate the serve.llm streaming overhead by properly batching stream chunks #### 2.46.0 (2025-05-07) - Add memory attribute to ExecutionResources - Support ray_remote_args for read_tfrecords - Add PhysicalOperator.min_max_resource_usage_bounds - Support num_cpus, memory, concurrency, batch_size for preprocess - Support new pyiceberg version - Lazily import torch FSDP for ray.train.torch module to improve performance and reduce unnecessary dependencies - Deserialize the user-defined training function directly on workers, improving efficiency - Refactor LLMServer and LLMEngine to not diverge too much from vllm chat formatting logic - Add router replicas and batch size to llm config - Bump vllm from 0.8.2 to 0.8.5 #### 2.45.0 (2025-04-29) - Make Object Store Fallback Directory configurable - Split dashboard single process into multiple processes to improve stability and avoid interference between different heads - Improve performance of sorting and sort-shuffle based operations by more than 5x #### 2.44.0 (2025-03-21) - Make chunk combination threshold configurable in Ray Data - Store average memory use per task in OpRuntimeMetrics - Avoid unnecessary conversion to Numpy when creating Arrow/Pandas blocks - Add append-mode API for preprocessors with vectorizers and hashers now outputting a single column instead of 1 column per feature - Fold v2.XGBoostTrainer API into the public trainer class as an alternate constructor - Create a default ScalingConfig if one is not provided to the trainer - Improve TrainingFailedError message in Ray Train - Clean up shutdown behavior of serve - Add additional_log_standard_attrs to serve logging config #### 2.43.0 (2025-02-27) - Implement accurate memory accounting for UnionOperator - Implement accurate memory accounting for all-to-all operations - Persist ParquetDatasource metadata - Pass executor as an argument to ExecutionCallback - Replace AggregateFn with AggregateFnV2 - Add state tracking to train v2 to make run status, run attempts, and training worker metadata observable #### 2.42.0 (2025-02-05) - Optimize multi-column groupbys in Ray Data - Include Ray user-agent in BigQuery client construction - Cache metrics in replica and report on an interval in Ray Serve - Cache expensive calls to inspect.signature in Ray Serve - Remove extra pickle serialization for gRPCRequest in Ray Serve - Implement shared LongPollClient for Routers in Ray Serve - Mark DeploymentHandle API as stable in Ray Serve - Enhance node-failure tolerance for new API stack in RLlib - Improve MetricsLogger cleanup throughput logic in RLlib - Split AddStates connectors into AddTimeDimToBatchAndZeroPad and AddStatesFromEpisodesToBatch in RLlib #### 2.41.0 (2025-01-23) - Use dask 2022.10.2 in Ray Data - Upgrade Arrow version from 17 to 18 in Ray Data - Update hudi version to 0.2.0 in Ray Data - Make select_columns and rename_columns use Project operator in Ray Data - Improved request_id generation to reduce proxy CPU overhead in Ray Serve - Tune GC threshold by default in proxy in Ray Serve - Use pickle.dumps for faster serialization from proxy to replica in Ray Serve #### 2.40.0 (2024-12-04) - Improve performance of DelegatingBlockBuilder - Improve memory accounting of pandas blocks - Handle multiple changed objects per LongPollHost.listen_for_change RPC in Ray Serve #### 2.39.0 (2024-11-13) - Marked num_rows_per_file parameter of write APIs as experimental - Dataset.count() now uses a Count logical operator - Upgrade to gymnasium==1.0.0 with support for new API for vector env resets - The distributed debugger can now be used without having to set RAY_DEBUG=1 #### 2.38.0 (2024-10-23) - Add SERVICE_UNAVAILABLE to list of retried transient errors - Re-phrase the streaming executor current usage string - Remove ray.kill in ActorPoolMapOperator - Simplify and consolidate progress bar outputs - Allow specifying both num_cpus and num_gpus for map APIs - Allow specifying insertion index when registering custom plan optimization Rules - PPO on new API stack now shuffles batches properly before each epoch - Switch on new API stack by default for SAC and DQN - Remove TensorFlow support on new API stack for PPO/IMPALA/APPO - Discontinue support for hybrid API stack - Store each GCS table in its own Redis Hash - Decouple create worker vs pop worker request #### 2.37.0 (2024-09-24) - Simplify custom metadata provider API - Change counts of metrics to rates of metrics - Refactor custom optimizer rules - Update run status and add stack trace to TrainRunInfo - Faster detection of dead replicas - Store episodes in state form for offline RL - Replace GAE in MARWILOfflinePreLearner with GeneralAdvantageEstimation connector in learner pipeline - Make is_gpu, is_actor, root_detached_id fields late bind to workers - Reconstruct actor to run lineage reconstruction triggered actor task #### 2.36.0 (2024-09-17) - Remove limit on number of tasks launched per scheduling step in Ray Data - Mark proxy as unready when its routers are aware of zero replicas in Ray Serve - Enhance offline RL in new API stack #### 2.35.0 (2024-08-28) - pip install ray[all] will no longer include ray[cpp] and the ray-cpp package; use pip install ray[cpp-all] to install everything including ray-cpp - Upgrade supported Arrow version from 16 to 17 - Various progress bar UX improvements in Ray Data - Try get size_bytes from metadata and consolidate metadata methods in Ray Data - Improve warning message when read task is large in Ray Data - Extend API to enable passing sample weights via ray.dataset.to_tf - Add a parameter to allow overriding LanceDB scanner options in Ray Data - Updates to support xgboost==2.1.0 in Ray Train - Improve handling of replica death and replica unavailability in deployment handle routers before controller restarts replica in Ray Serve #### 2.34.0 (2024-07-31) - Move DQN into the TargetNetworkAPI - Cleanup, rename, clarify: Algorithm.workers/evaluation_workers, local_worker(), etc. - Add new python GcsClient binding #### 2.33.0 (2024-07-25) - Add last exception to error message when GCS connection fails in ray.init() - Change offsets to int64 and change to LargeList for ArrowTensorArray - Prevent from_pandas from combining input blocks - Update Dataset.count() to avoid unnecessarily keeping BlockRefs in-memory - Use Set to fix inefficient iteration over Arrow table columns - Add AWS Error UNKNOWN to list of retried write errors - Always print traceback for internal exceptions - Allow unknown estimate of operator output bundles and ProgressBar totals - Improve filesystem retry coverage - Update run status and actor status for train runs #### 2.32.0 (2024-07-10) - Optimize ServeController.get_app_config() - Change default for max and target ongoing requests - Integrate Ray Serve with Ray structured logging - Allow configuring handle cache size and controller max concurrency - Optimize DeploymentDetails.deployment_route_prefix_not_set() - Allow env setup logger encoding - ray list tasks filter state and name on GCS side - Log ray version and ray commit during GCS start - Upgraded Dashboard to MUIv5 and React 18 - Make Tune trial ID available in EnvRunners and callbacks - Add env- and agent_steps to custom evaluation function - Remove default-metrics from Algorithm #### 2.31.0 (2024-06-26) - Missing stopping criterion should not error, just warn - Optimize DeploymentStateManager.get_deployment_statuses - Automatically infer worker path when starting worker in container #### 2.30.0 (2024-06-20) - Improve fractional CPU/GPU formatting - Use sampled fragments to estimate Parquet reader batch size - Refactor ParquetDatasource and metadata fetching logic - Refactor planner.py - Change the behavior of a missing stopping criterion metric to warn instead of raising an error - Enhance env-rendering callback - Integrate amdsmi in AMDAcceleratorManager - Parse out json logs in Dashboard #### 2.24.0 (2024-06-06) - Upgrade Arrow version to 16 in CI - Record more telemetry for newly added datasources - Improve excessive syncing warning - Clean up Serve proxy files - Improve node death observability - Clarify that num_rows_per_file is not strict #### 2.23.0 (2024-05-22) - Refine exception handling from arrow data conversion - Examples and tuned_examples learning test for new API stack are now self-executable and support WandB - Add runtime env started logs to job driver - ray.util.collective support torch.bfloat16 - Better propagate node death information #### 2.22.0 (2024-05-14) - Introduce abstract interface for data autoscaling - Add debugging info for SplitCoordinator - Update RayTrainReportCallback to only save checkpoints on rank 0 for XGBoost and LightGBM - Improve log viewer so line numbers do not get selected when copying text - Improve log viewer to avoid unnecessary re-rendering which causes text selection to clear #### 2.21.0 (2024-05-08) - Allow methods to pass type @serve.batch type hint - Allow configuring Serve control loop interval - Rename WorkerSet to EnvRunnerGroup, DEFAULT_POLICY_ID to DEFAULT_MODULE_ID, config.rollouts() to config.env_runners() - Change behavior of EnvRunnerGroup.foreach_worker methods to mark_healthy=True and healthy_only=True as new defaults #### 2.20.0 (2024-05-01) - Deduplicate repeated schema during ParquetDatasource metadata prefetching - Update map_groups implementation to better handle large outputs - Add default behavior to false for creating directories on S3 writes - Make internal UDF names more descriptive - Make name a required argument for AggregateFn - Setup XGBoost CommunicatorContext automatically - Remove trial table when running Ray Train in a Jupyter notebook - Clean up temporary checkpoint directories for class Trainables - Make handle push metric interval configurable with environment variable RAY_SERVE_HANDLE_METRIC_PUSH_INTERVAL_S - Improve performance of developer API serve.get_app_handle - Clean up AlgorithmConfig and rename properties and methods for better consistency and transparency - Report GCS internal pubsub buffer metrics and cap message size #### 2.12.0 (2024-04-25) - Remove spammy log for "new output engine" - Better support and separation-of-concerns for `model_config_dict` in new API stack - More example scripts got translated from the old- to the new API stack including Curriculum learning and custom-gym-env #### 2.11.0 (2024-04-17) - Pin ipywidgets==7.7.2 to enable Data progress bars in VSCode Web - Change log level for ignored exceptions - Change Parquet encoding ratio lower bound from 2 to 1 - Add support for new style lightning import #### 2.10.0 (2024-03-21) - Renamed max_concurrent_queries to max_ongoing_requests in Serve - Renamed target_num_ongoing_requests_per_replica to target_ongoing_requests in Serve - Renamed downscale_smoothing_factor to downscaling_factor in Serve - Renamed upscale_smoothing_factor to upscaling_factor in Serve - max_ongoing_requests (max_concurrent_queries) is now strictly enforced in Serve - Revamped XGBoostTrainer and LightGBMTrainer to no longer depend on xgboost_ray and lightgbm_ray - Refactored local staging directory to remove the need for local_dir and RAY_AIR_LOCAL_CACHE_DIR in Train/Tune #### 2.9.3 (2024-02-22) - Bump up task failure logs to warnings to make sure failures could be troubleshooted #### 2.9.2 (2024-02-06) - Improve stability and accuracy for Data+Train benchmark #### 2.9.1 (2024-01-19) - Enable setting queue length response deadline via environment variable #### 2.9.0 (2023-12-21) - Upgrade Ray Train support for spot node preemption to handle preemption node failures differently than application errors - Make Ray compatible with Pydantic versions <2.0.0 and >=2.5.0 - Support PyArrow 14.0.1 in Ray Data #### 2.8.1 (2023-12-01) - Migrate Logs page to use state api #### 2.8.0 (2023-11-03) - Update PyTorch Lightning import path to support both pytorch_lightning and lightning - Propagate driver DataContext to RayTrainWorkers - The Serve REST API is now accessible through the dashboard port which defaults to 8265 - Hard deprecate DatasetPipeline - Remove BulkExecutor code path - Remove legacy iteration code path in Ray Data #### 2.7.1 (2023-10-09) - Improve cluster page UI in Dashboard #### 2.7.0 (2023-09-17) - Rebrand Ray AI Runtime (AIR) to Ray AI Libraries and consolidate the ray.air namespace into ray.data, ray.train, and ray.tune - Read data with multi-threading for FileBasedDataSource in Ray Data - Add partition_filter parameter to read_parquet - Move away from LightningTrainer in favor of TorchTrainer as the recommended way of running distributed PyTorch Lightning #### 2.6.0 (2023-07-21) - Users are now required to provide cloud storage or NFS path for distributed training or tuning jobs instead of a local path - Enable execution optimizer by default in Ray Data - Enforce strict mode batch format for DataIterator.iter_batches() - Spread map task stages by default for arg size <50MB - Unify Lightning and AIR CheckpointConfig - Add support for custom pipeline class in TransformersPredictor #### 2.5.0 (2023-06-08) - Enable strict mode by default in Ray Data, requiring schemas for all Datasets and removing support for standalone Python objects - Set default batch format to NumPy in Ray Data for better performance in batch inference - Add RunConfig.storage_path to replace SyncConfig.upload_dir and RunConfig.local_dir - Use Ray storage URI as default storage path if configured - Use wait based prefetcher by default in Ray Data - Require compute spec to be explicitly spelled out #### 2.4.0 (2023-04-25) - Make Preprocessor.transform lazy by default - Make BatchPredictor lazy - Enhance logging and metrics for Serve applications - Replace ray_lightning integration with LightningTrainer for latest PyTorch Lightning compatibility #### 2.3.1 (2023-03-27) - Support different number of blocks/rows per block in zip() - Revert serve run to use Ray Client instead of Ray Jobs #### 2.3.0 (2023-02-24) - Rename Keras callback and its parameters to be more descriptive - Rename WandB save_checkpoints to upload_checkpoints - Rename DLPredictor.call_model tensor parameter to inputs - Lazy execution by default for Ray Datasets - Introduce streaming execution backend - Use Ray actors instead of multiprocessing for WandbLoggerCallback - Change default NCCL_SOCKET_IFNAME to blacklist veth - Change ScalingConfig to be optional for DataParallelTrainers if already in Tuner param_space #### 2.2.0 (2022-12-13) - Ray Dashboard now shows CPU flame graphs of Ray workers and new metrics for memory usage - Out-Of-Memory (OOM) Monitor is now enabled by default - Ray Data dynamic block splitting is now enabled by default to address out-of-memory and performance issues with large files #### 2.1.0 (2022-11-08) - Auto-enable GPU for Predictors - Deprecate Checkpoint.to_object_ref and Checkpoint.from_object_ref - Deprecate legacy train/tune functions in favor of Session #### 2.0.1 (2022-10-22) - Upgrade grpcio version to 1.32 - Upgrade redis version to 7.0.5 - Increase RPC timeout for dashboard - Replace deprecated name option with task_id in Workflows #### 2.0.0 (2022-08-23) - ray.ml is renamed to ray.air - ray.ml.preprocessors have been moved to ray.data - ray.ml.trainers have been moved to ray.train - ray.ml.predictors has been moved to ray.train ### Fixed #### 2.58.0 (2026-08-23) - Fix an RCE where read_lance or nested pickle objects could execute arbitrary code #### 2.56.1 (2026-07-17) - Fixed Arrow-backed to_pandas regressions by adding DataContext.enable_arrow_backed_pandas_conversion as an opt-out flag - Fixed int64/double[pyarrow] overflow crashes on concatenation by reconciling divergent numeric column types - Fixed TensorDtype.__from_arrow__ crash on zero-size tensor elements by using explicit row count instead of numpy dimension inference - Fixed crash in hash partition caused by read-only hash arrays - Nullified _input_dependencies in _get_args to prevent exponential sanitize_for_struct call chains over fused operators - Added protobuf >=7 compatibility to _proto_to_dict by binding to FieldDescriptor.is_repeated when the deprecated label attribute is absent - Fixed direct-streaming routing for body-aware routers by parsing raw request body into SimpleNamespace over routing-key fields - Disabled NCCL cuMem host buffer registration in CI pytests to stabilize GPU test runs - Upgraded ONNX example from MobileNet v1 to MobileNet v3 via torchvision and pinned onnxscript in GPU/ml-build CI dependency locks #### 2.55.1 (2026-04-22) - SSH connectivity issue in the ray-llm image #### 2.55.0 (2026-04-15) - Fix RCE in Arrow extension type deserialization from Parquet - Fix StreamingSplitDataIterator.schema() - Fix ParquetDatasource handling of FileSystemFactory.inspect - Fix read_parquet file-extension filtering for versioned object-store URIs - Fix wide_schema_pipeline_tensors cloudpickle deserialization - Fix OpBufferQueue race condition - Fix scheduling metrics computation - Fix OneHotEncoder max_categories to use global top-k instead of per-partition - Fix ReservationOpResourceAllocator resource borrowing for ActorPoolMapOperator - Fix DatabricksUCDatasource schema() shadowing by schema string attribute - Fix AliasExpr structural equality to respect rename flag - Fix _align_struct_fields failure with unaligned scalar fields - Fix min_scheduling_resources fallback to incremental_resource_usage - Fix output backpressure unblocking sequence for terminal ops - Fix multi-input operator object store memory attribution - Fix reference cycle by moving to module scope - Replace on_exit hook with __ray_shutdown__ to fix UDF cleanup race - Prevent Limit from getting pushed past map_groups - Propagate schema in empty _shuffle_block to fix ColumnNotFound in chained left joins - Clamp rolling utilization averages to zero - Fix floating point errors in TimeWindowAverageCalculator - Remove default task-level timeout and clamp end_offset in Kafka datasource - Avoid redundant reads in train_test_split - Return None when no outputs have been produced #### 2.54.1 (2026-03-25) - Disable hanging issue detection to prevent blocking calls to the Ray State API that could cause the scheduling loop to block and severely degrade pipeline performance #### 2.54.0 (2026-02-18) - Fuse MapBatches even if they modify the row count #### 2.53.0 (2025-12-20) - Fix support of list types for Unique aggregator - Fix reading from written parquet for numpy with NaNs #### 2.52.0 (2025-11-21) - Fixed renamed columns to be appropriately dropped from output - Fixed handling of renames in projection pushdown - Fixed broken LogicalOperator abstraction barrier in predicate pushdown rule #### 2.51.1 (2025-11-01) - Reuse previous metadata if transferring the same tensor list with nixl #### 2.51.0 (2025-10-29) - Fixed renamed columns to be appropriately dropped from output in Ray Data - Fixed driver hang during streaming generator block metadata retrieval in Ray Data - Fixed ControllerError triggered by after_worker_group_poll_status errors in Ray Train - Fixed iter_torch_batches use of ray.train.torch.get_device outside Train #### 2.50.1 (2025-10-18) - Fix deadlock when cancelling stale requests on in-order actors #### 2.50.0 (2025-10-10) - Fixed streaming executor to properly drain output queues - Fixed retry logic for hash shuffle operations - Fixed split_blocks to prevent producing empty blocks #### 2.49.1 (2025-09-03) - Fix issue where GPU metrics are missing in Ray Dashboard - Fix regression in handling very large schemas in Ray Data #### 2.49.0 (2025-08-26) - AutoscalingActorPool now properly downscales after execution #### 2.48.0 (2025-07-18) - Fixed map_groups issues - Prevented Op fusion for streaming repartition to avoid performance degradation - Fixed ActorPool autoscaler scaling up logic - Resolved empty dataset repartitioning issues #### 2.47.1 (2025-06-18) - Ray failed to start on Mac #### 2.47.0 (2025-06-12) - Make Ray Data from_torch respect Dataset len - Fix race condition bug in fault tolerance by disabling on_exit hook - Fix move_tensors_to_device utility for the list/tuple[tensor] case - Fix ActorPool scaling to avoid scaling down when the input queue is empty - Fix internal queues accounting for all Operators with an internal queue - Fix backpressure for FileBasedDatasource to prevent potential OOMs for workloads using FileBasedDatasources - Fixed GPU transfer support for non-contiguous tensors - Fixed runtime_env validation for py_modules - Disallow special characters in Serve deployment and application names - Add a descriptive error message when a deployment name is not found - LLMRouter.check_health() should check LLMServer.check_health() - Fix runtime passthrough and auto-executor class selection - Update check_health return type - Bug fix for duplication of token - In stream batching, first part of the stream was always consumed and not streamed back from the router - Fixed weight synching in offline evaluation - Fixed bug in split_and_zero_pad utility function related to complex structures vs simple values or np.arrays #### 2.46.0 (2025-05-07) - Handle Arrow Array null types in to_numpy - Fix S3 serialization wrapper compatibility with RetryingPyFileSystem - Fix Optimizer to apply rules until plan stabilizes - Fix FuseOperators rule to properly handle transformations that drastically change size of the dataset - Fix error when no arguments are passed into TorchTrainer #### 2.45.0 (2025-04-29) - Make sure KillActor RPC with force_kill=True can actually kill the threaded actor - Do not remove idle nodes for upcoming placement groups in Autoscaler - Prevent threaded actors from getting stuck forever when receiving two exit signals - Fix illegal memory access of cgraph when used in PP - Avoid resubmitted actor tasks from hanging indefinitely - Fix interleaved placement group creation process due to node failure - Flush task events in CoreWorker::Shutdown instead of CoreWorker::Disconnect #### 2.44.0 (2025-03-21) - Fix Map Operators to avoid unconditionally overriding generator's back-pressure configuration - Fix filter expr equating negative numbers in Ray Data - Fix StandardScaler to handle NaN stats in Ray Data #### 2.43.0 (2025-02-27) - Avoid memory leak with pyarrow.infer_type on datetime arrays - Fix parquet partition cols to support tensors types - Fix aggregation protocol to be appropriately associative #### 2.42.1 (2025-02-11) - Incorrect assertion in Ray Data #### 2.42.0 (2025-02-05) - Fix bug that made read tasks non-deterministic in Ray Data - Fix batched requests hanging after request cancellation bug in Ray Serve - Re-introduce mixin-replay-buffer pass for old API stack IMPALA/APPO even if replay-ratio=0 - Fix MetricsLogger race conditions in RLlib - Fix APPO/IMPALA bug for greater than 1 Learner actor in RLlib #### 2.41.0 (2025-01-23) - Fix groupby hang when value contains np.nan in Ray Data - Fix bug where file_extensions doesn't work with compound extensions in Ray Data #### 2.40.0 (2024-12-04) - Fix bug where you can't specify a schema with write_parquet - Fix bug where to_pandas errors if your dataset contains Arrow and pandas blocks - Fix bug where map_groups doesn't work with pandas data - Fix bug where write_parquet errors if your data contains nullable fields - Fix bug where "Iteration Blocked Time" charts looks incorrect - Fix bug where unique fails with null values - Fix bug where "Rows Outputted" is 0 in the Data dashboard - Fix bug where methods like drop_columns cause spilling - Fix bug where async map tasks hang - Fix StartTracebackWithWorkerRank serialization in Ray Train #### 2.39.0 (2024-11-13) - Fixed InvalidObjectError edge case with Dataset.split() - Made Concatenator preserve order of concatenated columns - transform_batch no longer fails on missing columns - Safely check if the storage filesystem is pyarrow.fs.S3FileSystem - Basic HTTP deployments will now return "Internal Server Error" instead of a traceback to match FastAPI behavior - Fixed an issue where high values of max_ongoing_requests couldn't be reached due to an interaction with core's max_concurrency - Fixed an edge case where pending requests were not canceled properly #### 2.38.0 (2024-10-23) - Fix bug where Ray Data incorrectly emits progress bar warning - Yield remaining results from async map_batches - Fix event loop mismatch with async map #### 2.37.0 (2024-09-24) - Throw exception for non-streaming HF datasets with override_num_blocks argument - Remove ineffective retry code in plan_read_op - Fix incorrect pending task size if outputs are empty - Fix component id logging field #### 2.36.1 (2024-09-23) - Fix broken dashboard cluster page when there are dead nodes - Fix broken dashboard worker page #### 2.36.0 (2024-09-17) - Display pending actors separately in the progress bar and not count them towards running resources - Fix bug where arrow_parquet_args aren't used - Skip empty JSON files in read_json() - Remove remote call for initializing Datasource in read_datasource() - Update TORCH_NCCL_ASYNC_ERROR_HANDLING env var - Allow get_serve_logs_dir to run outside of Ray's context - Fix large batch size for synchronous algos after EnvRunner failures - Fix ray_unintentional_worker_failures_total to only count unintentional worker failures #### 2.35.0 (2024-08-28) - Change type of DataContext.retried_io_errors from tuple to list in Ray Data - Change pickling log level from warning to debug in Ray Data - Add validation for shuffle arg in Ray Data - Fix validation bug when size=0 in ActorPoolStrategy in Ray Data - Fix exception in async map in Ray Data - Fix wrong metrics group for Object Store Memory metrics on Ray Data Dashboard - Handle errors in SplitCoordinator when generating a new epoch in Ray Data - Fix WandB metric overlap after restore from checkpoint in Tune #### 2.34.0 (2024-07-31) - Remove read task warning if size bytes not set in metadata - Sort workers by node ID rather than by node IP - Rename all np.product usage to np.prod for numpy compatibility #### 2.33.0 (2024-07-25) - Add object back to memory store when object recovery is skipped - Task status should start with PENDING_ARGS_AVAIL when retry - Fix ObjectFetchTimedOutError - Make working_dir support files created before 1980 - Allow full path in conda runtime env - Fix worker launch time formatting in state api - Replace lambda mutable default arguments in Ray Data #### 2.32.0 (2024-07-10) - Ensure InputDataBuffer doesn't free block references - MapOperator.num_active_tasks should exclude pending actors - Fix progress bars being displayed as partially completed in Jupyter notebooks - Decrement lineage ref count of an actor when the actor task return object reference is deleted #### 2.31.0 (2024-06-26) - Fixed bug where preserve_order doesn't work with file reads - Fix logging error on passing traceback object into exc_info - Run __del__ even if constructor is still in-progress - On GCS restart, destroy not forget the unused workers, fixing PG leaks - Cancel lease requests before returning a PG bundle - Fix boost fiber stack overflow #### 2.30.0 (2024-06-20) - Fix EMA stats without window that would lead to infinite list memory leak in MetricsLogger API - Fix the C++ GcsClient Del not respecting del_by_prefix - Fix exit handling of FiberState threads #### 2.24.0 (2024-06-06) - Avoid pickling LanceFragment when creating read tasks for Lance - DreamerV3 on tf bug fix to run again with tf==2.11.1 - Various MetricsLogger bug fixes - Fix worker crash when getting actor name from runtime context - Log dedup should not dedup number only lines - Move Job package uploading to another thread to unblock the event loop #### 2.23.0 (2024-05-22) - Fix Ray databricks UC reader with dynamic Databricks notebook scope token - Fix bug where you can't return objects and array from UDF - Fix bug where map_groups triggers execution during input validation - Fix PB2 scheduler error resulting from trying to sort by Trial objects - Log application unhealthy errors at error level instead of warning level - Fix result dict spam with duplicate and deprecated keys in top level - Fix worker process leaks after job finishes #### 2.22.0 (2024-05-14) - Don't show AllToAllOperator progress bar if the disable flag is set - Don't load Arrow PyExtensionType by default - Don't raise batch size error if num_gpus=0 - Fix the cpu percentage metrics for dashboard process #### 2.21.0 (2024-05-08) - Retry RaySystemError application errors - Fix bug with controller failing to recover for autoscaling deployments - Fix control+c after serve run doesn't shutdown serve components - Fix lightweight update max ongoing requests - Fix get_state() and from_state() methods in SingleAgent and MultiAgentEpisodes - Fix torch global_norm clipping overflow problem - Fix ray.init(logging_format) argument is ignored - Handle unserializable user exception - Fix dashboard process event loop blocking issues - Fix Nodes page sorting not working correctly - Add back actors per page UI control in the actors page #### 2.20.0 (2024-05-01) - Fix memory leak in handles for autoscaling deployments when RAY_SERVE_COLLECT_AUTOSCALING_METRICS_ON_HANDLE=1 - Fix task submission never returning when network partition happens - Fix incorrect use of SSH port forward option #### 2.12.0 (2024-04-25) - Fix the config to disable progress bar - Disable gathering the full state dict in `RayFSDPStrategy` for `lightning>2.1` - Fix getting attributes on stdout during Serve logging redirect - Fix GetAllJobInfo `is_running_tasks` is not returning the correct value when driver starts ray #### 2.11.0 (2024-04-17) - Fix throughput time calculations for metrics - Fix nested ragged numpy.ndarray - Fix Ray debugger incompatibility caused by trimmed error stack trace - Fix ScalingConfig(accelerator_type) to request an appropriate resource amount - Fix maximum recursion issue when serializing exceptions - Remove base config deepcopy when initializing the trainer actor - Dynamically-created applications will no longer be deleted when a config is PUT via the REST API - Fix _to_object_ref memory leak - Fix multi-GPU and multi-agent support on the new API stack #### 2.10.0 (2024-03-21) - Fix deadlocks caused by Dataset.streaming_split for job hanging #### 2.9.3 (2024-02-22) - Fix protobuf breaking change by adding a compat layer - Fix placement group leaks - Skip schema call in to_tf if tf.TypeSpec is provided - Skip recording memory spilled stats when get_memory_info_reply is failed (also in 2.9.2) - Fix DeploymentStateManager qualifying replicas as running prematurely #### 2.9.2 (2024-02-06) - Fix out of disk test on release branch - Fix failing huggingface test - Fix deadlocks caused by streaming_split - Fix locality config not being respected in DataConfig - Add retry for _sample_fragment during ParquetDatasource._estimate_files_encoding_ratio() - Pin the fastapi and starlette version to avoid breaking proxy - Fix IS_PYDANTIC_2 logic for pydantic<1.9.0 - Fix missing message body for json log formats #### 2.9.1 (2024-01-19) - Fix task events profile events per task leak - Make sure redis sync context and async context connect to the same redis instance - Retry write if error during file clean up - Improve handling the websocket server disconnect scenario - Fix pydantic config documentation #### 2.8.1 (2023-12-01) - Prevent state API from reading files outside the Ray log directory #### 2.8.0 (2023-11-03) - Fix child process leak issue from Ray worker that leaks GPU memory - Fix job page excessive loading time issue when Ray HA cluster restarts a head node - Fix bug where _StatsActor errors with PandasBlock #### 2.7.1 (2023-10-09) - Fix a bug where no data shows up on the `Error QPS per Application` panel in the Ray Dashboard - DreamerV3: Enable support for continuous actions - Fix a bug where setting a local storage path on Windows errors - Fix a broken `Trial.node_ip` property - Fix a segfault when a streaming generator and actor cancel is used together - Fix autoscaler SDK accidentally initializing a ray worker leading to leaked driver showing up in the dashboard - Fix bugs for the vSphere cluster launcher - Fix a bug where `ray start` would occasionally fail with `ValueError: `acceleratorType` should match v(generation)-(cores/chips).` - Fix a bug where overview page UI will crash in Dashboard - Fix Ray cluster stability on a high latency environment #### 2.6.3 (2023-08-15) - Fix memory leak from the end of object stream object in streaming generators - Fix serve run help message - Decrement ray_serve_deployment_queued_queries when client disconnects #### 2.6.2 (2023-08-03) - Pass logs through if sphinx-doctest is running - Pick GCP cluster launcher tests and fix - Apply request_timeout_s from Serve config to the cluster - Fix pyarrow lazy import #### 2.6.1 (2023-07-24) - Fix cluster launcher failing to start multi-node clusters in Ray 2.6.0 - Fix autoscaler env variable overwrite not being usable if the command itself uses the env variable - Fix Serve enum to_proto compatibility with Python 3.11 #### 2.6.0 (2023-07-21) - Fix infinite recursion in log redirection - Remove temporary checkpoint directories after restore - Fix bug in execution for actor re-use #### 2.5.1 (2023-06-21) - Don't error on eventual success when running with auto-recovery in Ray Train - Fix a bug that can cause undefined behavior when clusters attempt to scale up aggressively in Autoscaler - Fix mypy error where module "ray" does not explicitly export attribute "remote" #### 2.5.0 (2023-06-08) - Fix bugs in handling of nested ndarrays and other complex object types - Fix backpressure handling when reading directly from input datasource - Fix backpressure handling of queued actor pool tasks - Fix row count after applying filter #### 2.3.1 (2023-03-27) - Fix issue with max_concurrent_queries being ignored when autoscaling - Write Ray address even if Ray node is started with --block - Fix Ray on Spark running on layered virtualenv python environment - Fix disk metric showing double the actual value #### 2.3.0 (2023-02-24) - Fix and improve support for HDFS remote storage - Fix Imputer failing with categorical dtype #### 2.2.0 (2022-12-13) - Fix MLflow database integrity error in Ray AIR - Fix ResourceChangingScheduler dropping PlacementGroupFactory args in Ray AIR - Fix bug passing 'raise' to FailureConfig in Ray AIR - Fix reserved CPU warning if no CPUs are used in Ray AIR - Fix iter_batches() to not return empty batch in Ray Data #### 2.0.1 (2022-10-22) - Fix segfault when using runtime environments - Set correct path when using python -m - Fix autoscaling for 0 CPU head node - Allow code in private remote Git URIs to be imported in Serve - Fix hang during checkpoint synchronization in Tune - Fix trial restoration from different IP in Tune - Fix custom synchronizer serialization in Tune ### Removed #### 2.55.0 (2026-04-15) - Remove legacy BlockList, locality_with_output, old callback API, and PyArrow 9.0 checks #### 2.54.0 (2026-02-18) - Remove deprecated TENSOR_COLUMN_NAME constant #### 2.53.0 (2025-12-20) - Remove deprecated read_parquet_bulk API #### 2.52.0 (2025-11-21) - End of Life for Python 3.9 Support: Ray will no longer be releasing Python 3.9 wheels from now on #### 2.39.0 (2024-11-13) - Removed long-deprecated set_progress_bars - Removed deprecated API to set route_prefix per-deployment #### 2.36.0 (2024-09-17) - Remove support for nested DeploymentResponses #### 2.34.0 (2024-07-31) - Remove dead RayDatasetSpec #### 2.21.0 (2024-05-08) - Remove deprecated ray.air.callbacks modules #### 2.11.0 (2024-04-17) - Remove deprecated BatchPredictor - Remove scikit-optimize search algorithm - Remove batch_queue_cls parameter from the @serve.batch decorator #### 2.9.0 (2023-12-21) - Remove Legacy Trainers from Ray Train - Delete legacy TuneClient and TuneServer APIs from Ray Tune - Delete legacy Searchers from Ray Tune #### 2.8.0 (2023-11-03) - Remove deprecated Ray Serve 1.x APIs including Deployment.deploy, Deployment.delete, and Deployment.get_handle - Move 24 algorithms into rllib_contrib #### 2.6.0 (2023-07-21) - Remove ray.data.range_arrow() - Remove deprecated mlflow and wandb integrations ### Deprecated #### 2.58.0 (2026-08-23) - Deprecate ray_remote_args_fn and Dataset.zip #### 2.57.0 (2026-08-11) - Deprecate low-level scheduling APIs DataContext.scheduling_strategy, actor_locality_enabled, exclude_resources, and local:// ahead of the actor-only rearchitecture #### 2.56.0 (2026-06-29) - Deprecate ConcurrencyCapBackpressurePolicy, DataIterator.to_torch, and pandas UDF batches #### 2.53.0 (2025-12-20) - Ray plans to drop support for Pydantic V1 starting version 2.56.0 #### 2.44.0 (2025-03-21) - Ray Workflows library has been deprecated and will be removed in a future version of Ray #### 2.42.0 (2025-02-05) - Deprecate num_rows_per_file in favor of min_rows_per_file in Ray Data #### 2.41.0 (2025-01-23) - Default behavior for sync methods will change in a future release to run in a threadpool by default in Ray Serve #### 2.40.0 (2024-12-04) - Deprecate read_parquet_bulk - Deprecate iter_tf_batches - Deprecate meta_provider parameter of read functions - Deprecate to_torch #### 2.35.0 (2024-08-28) - Deprecate passing arguments that contain DeploymentResponses in nested objects to downstream deployment handle calls in Ray Serve #### 2.34.0 (2024-07-31) - Deprecate RLModuleWithTargetNetworksInterface #### 2.33.0 (2024-07-25) - Deprecate Dataset.get_internal_block_refs() #### 2.24.0 (2024-06-06) - Deprecate TUNE_RESULT_DIR, RAY_AIR_LOCAL_CACHE_DIR, and local_dir #### 2.20.0 (2024-05-01) - Deprecate prefetch_batches argument of iter_rows and change default value #### 2.11.0 (2024-04-17) - Log a deprecation warning for local_dir and related environment variables #### 2.8.0 (2023-11-03) - Python 3.7 support is officially deprecated from Ray - Deprecate previously experimental DAG API for deployment graphs in Ray Serve - Deprecate single-application config file format in Ray Serve - Deprecate accessing the Serve REST API through the dashboard agent port - Deprecate LightningTrainer, AccelerateTrainer, TransformersTrainer - Deprecate legacy DatasetConfig in Ray Train #### 2.7.0 (2023-09-17) - Ray Train Preprocessors and Predictors are deprecated in favor of Ray Data #### 2.6.0 (2023-07-21) - DatasetPipeline API is deprecated in favor of Dataset with streaming execution - BatchPredictor is deprecated #### 2.4.0 (2023-04-25) - Ray 2.4 is the last version that supports Python 3.6 #### 2.3.0 (2023-02-24) - Deprecate MlflowTrainableMixin in favor of setup_mlflow() function #### 2.2.0 (2022-12-13) - Deprecate Wandb mixin in Ray AIR - Deprecate Checkpoint.to_object_ref and Checkpoint.from_object_ref in Ray AIR ### Security #### 2.51.2 (2025-11-29) - reject Sec-Fetch-* other browser-specific headers in dashboard browser rejection logic for CVE-2025-62593 #### 2.52.1 (2025-11-28) - Improved handling for CVE-2025-62593 by testing for more browser-specific headers in dashboard browser rejection logic #### 2.52.0 (2025-11-21) - Token authentication: Ray now supports built-in token authentication across all components including the dashboard, CLI, API clients, and internal services #### 2.9.0 (2023-12-21) - Fix security vulnerabilities in Ray Dashboard (CVE-2023-6019, CVE-2023-6020, CVE-2023-6021, CVE-2023-48022, CVE-2023-48023) _2 releases carry no categorized changes yet: 2.49.2, 2.44.1._ ## Release notes ### 2.58.0 — Ray-2.58.0 - Date: 2026-08-23 - Version: 2.58.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.58.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.58.0 - **added** — Add Dataset.with_columns for multi-column expression projection - **added** — Add write_delta for Delta Lake with catalog support - **added** — Add Torch inference API - **added** — Promote hash shuffle v2 to a selectable shuffle strategy with aggregation support, vectorized aggregation, and block splitting during aggregation - **added** — Add ignore_missing_paths and skip_paths to read_parquet on DatasourceV2 - **added** — Add delta_timestamps (temporal windows) to read_lerobot - **added** — Collect cluster usage metrics by sampling in background threads during execution - **added** — Tolerate actor deaths during init via DataContext.max_consecutive_actor_init_deaths - **added** — Expose RAY_DATA_HASH_SHUFFLE_MAP_TASK_TARGET_INPUT_BYTES in DataContext - **added** — Add configurable status code and Retry-After header for backpressure rejections - **added** — Add dependency-ordered shutdown for deployments - **added** — Add an optional tie-break key to best-fit node scheduling - **added** — Scale ingress request router replicas per proxy node - **added** — Ray Train integrates with TorchTPU backend - **added** — Add a public preemption API and a controller PreemptingState - **added** — Add data ingest metrics to the Train dashboard - **added** — Add experimental Ray Sandbox which runs task and actor code under gVisor and can run Docker-built images directly - **added** — Add SubslicePlacementGroup for gang scheduling on TPU subslices and single-host TPU support in SlicePlacementGroup - **fixed** — Fix an RCE where read_lance or nested pickle objects could execute arbitrary code - **deprecated** — Deprecate ray_remote_args_fn and Dataset.zip #### Highlights * **Ray Serve LLM:** In this release we've completed KV cache and token aware request routing, which was previewed in 2.57. Tokenization now happens in-process on the `LLMRouter` ingress replica, the routing decision is made there, tokens are transmitted out-of-band so the engine does not re-tokenize, KV lifecycle events are broadcast to every ingress replica (\#64642, \#64920, \#64949, \#65010, \#65095). KV cache and token aware routing is also aware of CPU KV caches, so offloaded KV cache blocks count toward a replica's cache hit (\#65063). * **Ray Core:** We enabled the capability to offload task events from. With `RAY_enable_task_events_to_dashboard_head` on, the task event buffer is replaced by the ray event recorder, events are exported from the aggregator agent to a task events head that keeps an in-memory store, and the state APIs and `ray.timeline` read from it (\#64835, \#65028, \#65123, \#65160, \#65218). Enabling the feature removes task event ingestion and serving from the GCS hot path. * **Ray Data:** We’ve added Databricks integrations for writing to DeltaLake, with Catalog support. We’ve also shipped a new shuffle v2 backend, featuring improved performance for joins and aggregations. * **Sandboxing:** We've also added experimental Ray Sandbox, which runs task and actor code under gVisor and can run Docker-built images directly (\#64964, \#65570). * **TPU Support:** Ray Train adds support for TorchTPU backend (\#64796), and Ray Core adds `SubslicePlacementGroup` for gang scheduling on TPU subslices, single-host TPU support in `SlicePlacementGroup`, and resource accounting for `tpu7x` and multi-core chips (\#64578, \#64079, \#64058). This lets TPU slices and subslices be reserved and trained on without external gang-scheduling glue. #### Ray Data ###### 🎉 New Features * Add `Dataset.with_columns` for multi-column expression projection (\#63858) * Add `write_delta` for Delta Lake, with catalog support (\#64923, \#65079) * Add Torch inference API (\#65157) * Promote hash shuffle v2 to a selectable shuffle strategy, with aggregation support, vectorized aggregation, and block splitting during aggregation (\#64953, \#64652, \#64956, \#65329, \#64897) * Add `ignore_missing_paths` and `skip_paths` to `read_parquet` on DatasourceV2 (\#65118) * Add `delta_timestamps` (temporal windows) to `read_lerobot` (\#64877) * Collect cluster usage metrics by sampling in background threads during execution (\#64686) * Tolerate actor deaths during init via `DataContext.max_consecutive_actor_init_deaths` (\#64846) * Expose `RAY_DATA_HASH_SHUFFLE_MAP_TASK_TARGET_INPUT_BYTES` in `DataContext` (\#65103) ###### 💫 Enhancements * Generate sortable, collision-resistant dataset IDs (\#65075) * Push `Limit` into `ReadFiles` when it sits directly on it, and extract `FileIndexer.list_file_infos` (\#65167, \#65168) * Use `BlockRefCounter` for object store memory estimation and remove `BlockRefCounter.clear()` (\#64456, \#64521) * Fail execution if no operator makes progress within a timeout, instead of hanging (\#65349) * Improve `BatchIterator` and `iter_torch_batches`, and allow a custom `collate_fn` with a custom device (\#64994, \#64967) * Skip downstream-capacity backpressure for eligible materializers (\#64844) * Shard exact download partitioning (\#64066) * Bump the `sort_reduce` memory multiplier to 3x and stop using `estimated_input_blocks` as the shuffle partition count (\#65176, \#65296, \#65335) * Disable cluster autoscaling when `PlacementGroupSchedulingStrategy` is in use (\#64417) * Propagate the operator name to shuffle tasks (\#64802) * Rename `reports_custom_op_stats` to `should_report_custom_op_stats` across all `MapTransformFn` variants (\#64461, \#64515) * Rename `allocated` to `reserved` and add type aliases in the autoscaling coordinator (\#64997, \#65096) * Deprecate `ray_remote_args_fn` and `Dataset.zip` (\#64963, \ _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.58.0]_ ### 2.57.0 — Ray-2.57.0 - Date: 2026-08-11 - Version: 2.57.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.57.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.57.0 - **added** — Enable DataSourceV2 by default via DataContext.use_datasource_v2 - **added** — Add task-based hash shuffle v2 with ShuffleMapOp, ShuffleReduceOp, join support, multi-input reduce, downstream map fusion, and reducer remote args - **added** — Add Catalog abstraction with UnityCatalog implementation for read operations and Unity Catalog write support for Parquet and Iceberg - **added** — Add read_zarr for Zarr datasets - **added** — Add read_lerobot for LeRobot v3 datasets - **added** — Add PushdownCountFiles optimization to answer count() from Parquet footers - **added** — Add common subexpression elimination to the expression optimizer - **added** — Add GPU support for Aggregate - **added** — Make dataset iteration metrics queryable per split - **added** — Add custom operator stats to capture worker-side metrics during task execution - **added** — Add embedded RocksDB storage backend for GCS fault tolerance selectable with RAY_gcs_storage=rocksdb and RAY_gcs_storage_path - **added** — Add public API for topology-aware scheduling - **added** — Add gRPC support to HAProxy direct-ingress path including streaming, metrics, and custom request IDs - **added** — Add experimental KV-cache-aware request routing that tracks replica KV state, tokenizes before routing, and routes on prefill/decode token load - **changed** — Hash Shuffle V2 replaces aggregator actor pool with two stateless task-based operators that pass shards through the object store - **changed** — Distribute HAProxy ingress as the ray-haproxy PyPI package instead of compiling into images, making it the default HAProxy binary - **changed** — Default read_numpy to allow_pickle=False and make it manually configurable - **changed** — Make write_lance with mode=CREATE error instead of silently overwriting - **deprecated** — Deprecate low-level scheduling APIs DataContext.scheduling_strategy, actor_locality_enabled, exclude_resources, and local:// ahead of the actor-only rearchitecture #### Highlights * **Ray Data:** In this Ray release, we've enabled `DataSourceV2` by default (#64821), so `read_parquet` and friends use the new scan/listing infrastructure with row-group-aware chunking and predicate splitting. Hash Shuffle V2 eliminates the aggregator actor pool. V1 had to provision that pool up front from an estimate of the input size, and its actors accumulated partition shards in actor heap memory, invisible to Ray and unspillable, until finalization. V2 replaces it with two stateless task-based operators, ShuffleMapOp --> ShuffleReduceOp, that pass shards through the object store, so intermediate state spills under pressure and no capacity has to be reserved in advance. The map/reduce barrier itself remains in both designs. * HashShuffleV2 supports `join` ([#63598](https://github.com/ray-project/ray/pull/63598), [#64538](https://github.com/ray-project/ray/pull/64538), [#64687](https://github.com/ray-project/ray/pull/64687)). This lets shuffles reuse standard map/reduce scheduling, backpressure, and resource accounting. * **Ray Serve:** The HAProxy ingress is now distributed as the `ray-haproxy` PyPI package instead of being compiled into images, and it is the default HAProxy binary (#64141, #64163, #64164). We've also added gRPC support to the HAProxy direct-ingress path, including streaming, metrics, and custom request IDs (#63735, #64310, #64166, #64112). For Ray Serve LLM, we've added experimental KV-cache-aware request routing that tracks replica KV state through an event plane, tokenizes before routing, and routes on prefill/decode token load (#64084, #64085, #64097, #64224, #64327, #64400). KV cache-aware routing’s complete support will land in 2.58. * **Ray Core:** We've added an embedded RocksDB storage backend for GCS fault tolerance (REP-64), selectable with `RAY_gcs_storage=rocksdb` and `RAY_gcs_storage_path` (#63657). GCS fault tolerance no longer requires an external Redis instance. We've also added a public API for topology-aware scheduling (#63479, #63740). #### Ray Data ###### 🎉 New Features * Enable `DataSourceV2` by default via `DataContext.use_datasource_v2` (#64821) * New task-based hash shuffle v2 (`ShuffleMapOp` → `ShuffleReduceOp`) with `join`, multi-input reduce, downstream map fusion, and reducer remote args, behind an env flag (#63598, #64538, #64687, #64438, #64302, #64532, #64481) * Add a `Catalog` abstraction with a `UnityCatalog` implementation that can be passed to `read_*`, and Unity Catalog write support for Parquet and Iceberg (#64193, #64519) * Add `read_zarr` for Zarr datasets (#63003) and `read_lerobot` for LeRobot v3 datasets (#63821) * Add `PushdownCountFiles` optimization to answer `count()` from Parquet footers (#64763) * Add common subexpression elimination to the expression optimizer (#63974) * Add GPU support for `Aggregate` (#63708) * Make dataset iteration metrics queryable per split (#64608) * Add custom operator stats to capture worker-side metrics during task execution (#64221) * Refactor usage collection into an extensible `UsageCallback` (#64500) * Export `from_blocks` from `ray.data` (#64127) * Add support for `rapidsmpf-26.4.0` (#64324) ###### 💫 Enhancements * Hide Ray Data internal frames from user-code error tracebacks (#64587) * Run projection/predicate pushdown before limit pushdown, block optimization of non-deterministic expressions, and make fuse checks consistent between the logical and physical optimizers (#64651, #64165, #63936) * Add a two-mode block metadata fetch behind a single `MetadataFetcher` interface (#64378) * Use work stealing for file listing and a dynamic work queue for traversals (#64675, #64388) * Speed up `ExecutionResources` and the reservation/budget loops on the scheduling hot path (#63964) * Fix `O(n^2)` schema reconciliation in `unify_schemas` and avoid per-column `Series` materialization in tensor-column casting (#64555, #64038) * Decouple collate and memory pinning in `iter_torch_batch _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.57.0]_ ### 2.56.1 — Ray-2.56.1 - Date: 2026-07-17 - Version: 2.56.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.56.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.56.1 - **fixed** — Fixed Arrow-backed to_pandas regressions by adding DataContext.enable_arrow_backed_pandas_conversion as an opt-out flag - **fixed** — Fixed int64/double[pyarrow] overflow crashes on concatenation by reconciling divergent numeric column types - **fixed** — Fixed TensorDtype.__from_arrow__ crash on zero-size tensor elements by using explicit row count instead of numpy dimension inference - **fixed** — Fixed crash in hash partition caused by read-only hash arrays - **fixed** — Nullified _input_dependencies in _get_args to prevent exponential sanitize_for_struct call chains over fused operators - **fixed** — Added protobuf >=7 compatibility to _proto_to_dict by binding to FieldDescriptor.is_repeated when the deprecated label attribute is absent - **fixed** — Fixed direct-streaming routing for body-aware routers by parsing raw request body into SimpleNamespace over routing-key fields - **changed** — The threshold memory monitor now snapshots both user and system cgroup slices and logs an error when system-slice usage exceeds reserved system memory - **fixed** — Disabled NCCL cuMem host buffer registration in CI pytests to stabilize GPU test runs - **fixed** — Upgraded ONNX example from MobileNet v1 to MobileNet v3 via torchvision and pinned onnxscript in GPU/ml-build CI dependency locks #### Highlights - **Ray Data**: We added fixes for several `to_pandas` regressions introduced in 2.56: an opt-out flag (`RAY_DATA_ENABLE_ARROW_BACKED_PANDAS_CONVERSION`) for Arrow-backed conversion, an int64/`double[pyarrow]` overflow crash on concatenation, and a `TensorDtype.__from_arrow__` crash on empty tensor columns (#64793, #64794). - **Ray Core**: We added early detection for system-slice memory pressure: the memory monitor now snapshots the user and system cgroup slices together and logs an error when the system slice exceeds reserved system memory, warning users to raise `--system-reserved-memory` before it causes node deaths (#64492). - **Ray Serve**: We added protobuf 7 compatibility and a routing fix for LLM direct streaming, so body-aware routers like `PrefixCacheAffinityRouter` no longer hang when `RAY_SERVE_LLM_ENABLE_DIRECT_STREAMING=1` (#64592, #64488). #### Ray Data ##### 🔨 Fixes - Fixed two Arrow-backed `to_pandas` regressions: added `DataContext.enable_arrow_backed_pandas_conversion` as an opt-out, and reconciled divergent numeric column types before concatenation to avoid int64/`double[pyarrow]` overflow crashes (#64793, #64768). - Fixed a `TensorDtype.__from_arrow__` crash on zero-size tensor elements by using an explicit row count instead of numpy's `-1` dimension inference (#64794, #64767). - Fixed a crash in hash partition caused by read-only hash arrays (#64584, #64552, #64559). - Nullified `_input_dependencies` in `_get_args` so exporting operator args no longer triggers an exponential `sanitize_for_struct` call chain over fused operators (#64412, #64316). #### Ray Serve ##### 🔨 Fixes - Added protobuf `>=7` compatibility to `_proto_to_dict` by binding to `FieldDescriptor.is_repeated` when the deprecated `label` attribute is absent (#64592, #64362). #### Ray LLM ##### 🔨 Fixes - Fixed direct-streaming routing for body-aware routers: the ingress now parses the raw request body into a `SimpleNamespace` over routing-key fields (`messages`, `prompt`) so `choose_replica` receives the message body instead of raw bytes (#64488, #64328, #64326). #### Ray RLlib ##### 🔨 Fixes - Upgraded the ONNX example from the retired MobileNet v1 to MobileNet v3 via `torchvision` and pinned `onnxscript` in the GPU/ml-build CI dep locks, fixing ONNX export failures (#64591, #64028, #64031, #64590, #64033). #### Ray Core ##### 💫 Enhancements - The threshold memory monitor now snapshots both the user and system cgroup slices and logs an error when system-slice usage exceeds the reserved system memory, prompting users to raise `--system-reserved-memory` (#64492). ##### 🔨 Fixes - Disabled NCCL `cuMem` host buffer registration in CI pytests to stabilize GPU test runs (#64580, #64146). #### Documentation ##### 📖 Documentation - Repointed the ASHA Tune example links to the renamed `README` document, fixing the `fail_on_warning` ReadTheDocs build on the release line (#64761, #64630). - Updated the Python 3.10 CPU `pip freeze` dependency list for the Ray 2.56.0 release (#64447, #64357). #### Thanks Many thanks to all those who contributed to this release! @eicherseiji, @ryankert01, @bveeramani, @elliot-barn, @iamjustinhsu, @Sparks0219, @owenowenisme, @dstrodtman, @marwan116 ### 2.56.0 — Ray-2.56.0 - Date: 2026-06-29 - Version: 2.56.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.56.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.56.0 - **added** — Support multiple datasets per cluster via subcluster labels and resource partitioning - **added** — Add Dataset.mix() public API and MixOperator for weighted dataset mixing - **added** — Implement new DataSourceV2 framework with ParquetDatasourceV2, chunked reader, predicate splitting, and listing/scanner infrastructure - **added** — Add batch_size='auto' to map_batches to derive batch row count from target row batch size - **added** — Implement distributed upsert for Iceberg using task-based merge algorithm - **added** — Add include_row_hash to read_parquet - **added** — Add JAX data iterator - **added** — Expose isolate_read_workers flag to run read tasks on isolated worker processes - **added** — Expose default_map_logical_memory_enabled flag to set default logical memory for map operators - **added** — Support predicate pushdown for Lance format - **added** — Support per-partition start_offset and end_offset for read_kafka - **added** — Add obstore async download backend for download operator - **added** — Support UDF retries on transient exceptions - **added** — Add GPU-domain-aware placement groups using label locality for packing bundles onto nodes with ray.io/gpu-domain label - **added** — Add initial Kubernetes in-place pod resizing support for Autoscaler v2 - **added** — Introduce session-sticky routing via consistent hashing with ConsistentHashRouter - **added** — Add CapacityQueueRouter for supply-constrained workloads - **changed** — Fix iter_batches spilling by replacing make_async_gen with iter_threaded and reducing buffered batches - **changed** — Re-architect Ray Serve LLM by decoupling request handling from token streaming response path - **deprecated** — Deprecate ConcurrencyCapBackpressurePolicy, DataIterator.to_torch, and pandas UDF batches #### Highlights * **Ray Data Stability:** In this Ray release, we've added a variety of stability improvements, including running multiple datasets in a cluster, adding automatic batch size selection to CPU-based map-batches, and default logical memory configuration to prevent OOMs. We've also tightened `iter_batches` stability by reducing hidden buffering and shutting down the executor when consumers exit early (#63660, #63682, #62949). This reduces object-store spilling for common training workloads * **Ray Serve:** We re-architected Ray Serve LLM by decoupling request handling from token streaming response path (#62667, #62680, #62668, #62669, #63167), resulting in significant LLM serving performance improvements. We've also introduced new routing policies such as session-sticky routing via consistent hashing with `ConsistentHashRouter` (#62905, #63096, #62906) and `CapacityQueueRouter` (#62323) which is beneficial for supply-constrained workloads. * **Ray Core:** We've added GPU-domain-aware placement groups using label locality (#61442, #61614, #62487, #62533). This enables placement groups to pack bundles onto nodes that share a `ray.io/gpu-domain` label instead of only packing at the single-node level. We've also added initial Kubernetes in-place pod resizing support for Autoscaler v2 (#55961, #62369, #62215), enabling Ray clusters to resize CPU and memory on existing worker pods before scaling out new pods. #### Ray Data ###### 🎉 New Features * Support multiple datasets per cluster via subcluster labels and resource partitioning (#63331, #63375, #63982) * Add `Dataset.mix()` public API and `MixOperator` for weighted dataset mixing (#63168, #62450) * New DataSourceV2 framework: `ParquetDatasourceV2`, chunked reader, predicate splitting, listing/scanner infra (#63113, #63454, #63163, #62975, #63027, #62182) * Add `batch_size='auto'` to `map_batches` to derive batch row count from target row batch size (#62648) * Implement distributed upsert for Iceberg using task-based merge algorithm, preventing performance bottleneck on driver (#63482) * Add `include_row_hash` to `read_parquet` (#61408) * Add JAX data iterator (#61630) * Expose flag to run read tasks on isolated worker processes via `isolate_read_workers` (#63490) * Expose flag to set default logical memory for map operators via `default_map_logical_memory_enabled` (#63814) * Support predicate pushdown for Lance format (#61400) * Support per-partition `start_offset` and `end_offset` for `read_kafka` (#61620) * Add obstore async download backend for download operator (#61735) * Support UDF retries on transient exceptions (#63023) ###### 💫 Enhancements * Fix `iter_batches` spilling by replacing `make_async_gen` with `iter_threaded` and reducing buffered batches (#63660, #63682) * Gate `restore_original_order` in `iter_batches` behind `preserve_order` (#63792) * Convert `drop_columns` to a `Project` logical operator when input schema is known (#63813) * Make `ConcatAggregation` and `TurbopufferDatasink` use `polars` for sorting (#61904) * Boost and vectorize `hash_partition` with `sort_indices`, zero-copy slices, and pandas (#63498, #62757, #63152, #62587) * Enable `GPU_SHUFFLE` in `grouped_data.py` (#62410) * Eager `StarExpr` expansion, schema inference for non-black-box UDFs, and Expressions struct support (#63776, #63387, #62560) * Make logging configurable via `RAY_DATA_LOG_LEVEL` and log `RAY_DATA` env vars at execution start (#63487, #63380) * Display and track logical memory in progress bar (#63379) * Honor `compute=` in `filter(expr=...)` and deprecate `concurrency=` (#63576) * Enable filter pushdown through `StreamingRepartition` and read stage column-rename removal (#62347, #63384, #63582) * Cache deserialized Arrow schemas in `BlockMetadataWithSchema` (#63462) * Track scheduling-loop step duration (p50/p90/max), peak USS/object-store memory, and task block locality (#63586, #63345, #63489, #63418, #62249) * Replace `Ta _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.56.0]_ ### 2.55.1 — Ray-2.55.1 - Date: 2026-04-22 - Version: 2.55.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.55.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.55.1 - **fixed** — SSH connectivity issue in the ray-llm image - **changed** — Upgrade apt packages in slim base - Fixes SSH connectivity issue in the `ray-llm` image (#62625 / #62718). - Upgrade apt packages in slim base (#62666 / #62717). ### 2.55.0 — Ray-2.55.0 - Date: 2026-04-15 - Version: 2.55.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.55.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.55.0 - **added** — Add DataSourceV2 API with scanner/reader framework, file listing, and file partitioning - **added** — Support GPU shuffle with rapidsmpf 26.2 - **added** — Add Kafka datasink, migrate to confluent-kafka, support datetime offsets - **added** — Add Turbopuffer datasink - **added** — Add 2-phase commit checkpointing with trie recovery and load method - **added** — Integrate queue-based autoscaling policy with task consumers - **added** — Enable autoscaling for GPU stages - **added** — Add random(), uuid(), cast, and map namespace support to Expressions - **added** — Add support for Arrow native fixed-shape tensor type - **added** — Support writing tensors to tfrecords - **added** — Add pathlib.Path support to read_* functions - **added** — Add cudf as a batch_format - **added** — Allow ActorPoolStrategy for read_datasource() via compute parameter - **added** — Introduce ExecutionCache for streamlined caching - **added** — Support strict=False mode for StreamingRepartition - **added** — Add monotonically increasing ID support - **changed** — Cache _map_task args, improve heap-based actor ranking, and improve actor pool map - **changed** — Optimize concat tables and PyArrow schema hashing - **changed** — Reduce default DownstreamCapacityBackpressurePolicy threshold to 50% - **changed** — Improve reproducibility for random APIs - **changed** — Port changes from lance-ray into Ray Data - **changed** — Enable PyArrow compute-to-expression conversion for predicate pushdown - **changed** — Upgrade to pyiceberg 0.11.0 and cap pandas to <3 - **changed** — Refactor logical operators to frozen dataclasses - **fixed** — Fix RCE in Arrow extension type deserialization from Parquet - **fixed** — Fix StreamingSplitDataIterator.schema() - **fixed** — Fix ParquetDatasource handling of FileSystemFactory.inspect - **fixed** — Fix read_parquet file-extension filtering for versioned object-store URIs - **fixed** — Fix wide_schema_pipeline_tensors cloudpickle deserialization - **fixed** — Fix OpBufferQueue race condition - **fixed** — Fix scheduling metrics computation - **fixed** — Fix OneHotEncoder max_categories to use global top-k instead of per-partition - **fixed** — Fix ReservationOpResourceAllocator resource borrowing for ActorPoolMapOperator - **fixed** — Fix DatabricksUCDatasource schema() shadowing by schema string attribute - **fixed** — Fix AliasExpr structural equality to respect rename flag - **fixed** — Fix _align_struct_fields failure with unaligned scalar fields - **fixed** — Fix min_scheduling_resources fallback to incremental_resource_usage - **fixed** — Fix output backpressure unblocking sequence for terminal ops - **fixed** — Fix multi-input operator object store memory attribution - **fixed** — Fix reference cycle by moving to module scope - **fixed** — Replace on_exit hook with __ray_shutdown__ to fix UDF cleanup race - **fixed** — Prevent Limit from getting pushed past map_groups - **fixed** — Propagate schema in empty _shuffle_block to fix ColumnNotFound in chained left joins - **fixed** — Clamp rolling utilization averages to zero - **fixed** — Fix floating point errors in TimeWindowAverageCalculator - **fixed** — Remove default task-level timeout and clamp end_offset in Kafka datasource - **fixed** — Avoid redundant reads in train_test_split - **fixed** — Return None when no outputs have been produced - **removed** — Remove legacy BlockList, locality_with_output, old callback API, and PyArrow 9.0 checks ##### Ray Data ###### 🎉 New Features - Add `DataSourceV2` API with scanner/reader framework, file listing, and file partitioning (#61220, #61615, #61997) - Support GPU shuffle with `rapidsmpf` 26.2 (#61371, #62062) - Add Kafka datasink, migrate to `confluent-kafka`, support `datetime` offsets (#60307, #61284, #60909) - Add Turbopuffer datasink (#58910) - Add 2-phase commit checkpointing with trie recovery and load method (#61821, #60951) - Queue-based autoscaling policy integrated with task consumers (#59548, #60851) - Enable autoscaling for GPU stages (#61130) - Expressions: add `random()`, `uuid()`, `cast`, and map namespace support (#59656, #60695, #59879) - Add support for Arrow native fixed-shape tensor type (#56284) - Support writing tensors to tfrecords (#60859) - Add `pathlib.Path` support to `read_*` functions (#61126) - Add `cudf` as a `batch_format` (#61329) - Allow `ActorPoolStrategy` for `read_datasource()` via `compute` parameter (#59633) - Introduce `ExecutionCache` for streamlined caching (#60996) - Support `strict=False` mode for `StreamingRepartition` (#60295) - Port changes from lance-ray into Ray Data (#60497) - Enable PyArrow compute-to-expression conversion for predicate pushdown (#61617) - Add vLLM metrics export and Data LLM Grafana dashboard (#60385) - Include logical memory in resource manager scheduling decisions (#60774) - Add monotonically increasing ID support (#59290) ###### 💫 Enhancements - Performance: cache `_map_task` args, heap-based actor ranking, actor pool map improvements (#61996, #62114, #61591) - Optimize concat tables and PyArrow schema hashing (#61315, #62108) - Reduce default `DownstreamCapacityBackpressurePolicy` threshold to 50% (#61890) - Improve reproducibility for random APIs (#59662) - Clamp batch size to fall within C++ 32-bit int range (#62242) - Account for external consumer object store usage in resource manager budget (#62117) - Make `get_parquet_dataset` configurable in number of fragments to scan (#61670) - Consolidate schema inference and make all preprocessors implement `SerializablePreprocessorBase` (#61213, #61341) - Disable hanging issue detection by default (#62405) - Make execution callback dataflow explicit to prevent state leakage (#61405) - Log `DataContext` in JSON format at execution start for traceability (#61150, #61428) - Autoscaler: configurable traceback, Prometheus gauges, relaxed constraints (#62210, #62209, #61917, #61385) - Add metrics for task scheduling time, output backpressure, and logical memory (#61192, #61007, #61436) - Prevent operators from dominating entire shared object store budget (#61605) - Eliminate generators to avoid intermediate state pinning (#60598) - Default log encoding to UTF-8 on Windows (#61143) - Remove legacy `BlockList`, `locality_with_output`, old callback API, PyArrow 9.0 checks (#60575, #61044, #62055, #61483) - Upgrade to `pyiceberg` 0.11.0; cap `pandas` to <3 (#61062, #60406) - Refactor logical operators to frozen dataclasses (#61059, #61308, #61348, #61349, #61351, #61364, #61481) - Prevent aggregator head node scheduling (#61288) - Add error for `local://` paths with a zero-resource head node (#60709) ###### 🔨 Fixes - Fix RCE in Arrow extension type deserialization from Parquet (#62056) - Fix `StreamingSplitDataIterator.schema()` (#62057) - Fix `ParquetDatasource` handling of `FileSystemFactory.inspect` (#62065) - Fix `read_parquet` file-extension filtering for versioned object-store URIs (#61376) - Fix `wide_schema_pipeline_tensors` cloudpickle deserialization (#62149) - Fix `OpBufferQueue` race condition (#60828) - Fix scheduling metrics computation (#62031) - Fix `OneHotEncoder` `max_categories` to use global top-k instead of per-partition (#60790) - Fix `ReservationOpResourceAllocator` resource borrowing for `ActorPoolMapOperator` (#60882) - Fix `DatabricksUCDatasource` `schema()` shadowing by schema string attribute (#61282) - Fix `AliasExpr` st _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.55.0]_ ### 2.54.1 — Ray-2.54.1 - Date: 2026-03-25 - Version: 2.54.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.54.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.54.1 - **fixed** — Disable hanging issue detection to prevent blocking calls to the Ray State API that could cause the scheduling loop to block and severely degrade pipeline performance ##### Ray Data 🔨 Fixes - Disable hanging issue detection (#61895) — The hanging issue detector was making blocking calls to the Ray State API, which could cause the scheduling loop to block and severely degrade pipeline performance. The detector is disabled in this patch release until the blocking calls are fixed. ### 2.54.0 — Ray-2.54.0 - Date: 2026-02-18 - Version: 2.54.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.54.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.54.0 - **added** — Add checkpointing support to Ray Data - **added** — Add list operations, fixed-size arrays, string padding, logarithmic, trigonometric, arithmetic, and rounding to Compute Expressions - **added** — Add sql_params support to read_sql - **added** — Add AsList aggregation - **added** — Support CountDistinct aggregate - **added** — Add credential provider abstraction for Databricks UC datasource - **added** — Support callable classes for UDFExpr - **added** — Add autoscaler metrics to Data Dashboard - **added** — Add optional filesystem parameter to download expression - **added** — Allow specifying partitioning style or flavor in write_parquet() - **added** — Enable new cluster autoscaler by default - **added** — Add Queue-based autoscaling for TaskConsumer deployments with QueueMonitor actor support for Redis and RabbitMQ - **changed** — Improve numerical stability in scalers by handling near-zero values - **changed** — Update to PyArrow 23 - **changed** — Demote Sort from requiring preserve_order - **changed** — Use Arrow IPC for Arrow Schema serialization and deserialization - **changed** — Enable and tune DownstreamCapacityBackpressurePolicy - **changed** — Decouple Ray Train from Ray Data by removing top-level ray.data imports - **changed** — Move extension types to ray.data - **fixed** — Fuse MapBatches even if they modify the row count - **removed** — Remove deprecated TENSOR_COLUMN_NAME constant ##### Ray Data 🎉 New Features - Add checkpointing support to Ray Data (#59409) - Compute Expressions: list operations (#59346), fixed-size arrays (#58741), string padding (#59552), logarithmic (#59549), trigonometric (#59712), arithmetic (#59678), and rounding (#59295) - Add `sql_params` support to `read_sql` (#60030) - Add `AsList` aggregation (#59920) - Support `CountDistinct` aggregate (#59030) - Add credential provider abstraction for Databricks UC datasource (#60457) - Support callable classes for `UDFExpr` (#56725) - Add autoscaler metrics to Data Dashboard (#60472) - Add optional filesystem parameter to download expression (#60677) - Allow specifying partitioning style or flavor in `write_parquet()` (#59102) - New cluster autoscaler enabled by default (#60474) 💫 Enhancements - Improve numerical stability in scalers by handling near-zero values (#60488) - Export dataset operator output schema to event logger (#60086) - Iceberg: add retry policy for Storage + Catalog writes (#60620) - Iceberg: remove calls to Catalog Table in write tasks (#60476) - Expose logical operators and rules via package exports (#60297, #60296) - Demote Sort from requiring `preserve_order` (#60555) - Improve appearance of repr(dataset) (#59631) - Allow configuring `DefaultClusterAutoscalerV2` thresholds via env vars (#60133) - Use Arrow IPC for Arrow Schema serialization/deserialization (#60195) - Store _source_paths in object store to prevent excessive spilling during read task serialization (#59999) - Add more shuffle fusion rules (#59985) - Enable and tune `DownstreamCapacityBackpressurePolicy` (#59753) - Enable concurrency cap backpressure with tuning (#59392) - Set default actor pool scale up threshold to 1.75 (#59512) - Don't downscale actors if the operator hasn't received any inputs (#59883) - Don't reserve GPU budget for non-GPU tasks (#59789) - Only return selected data columns in hive-partitioned Parquet files (#60236) - Ordered + FIFO bundle queue (#60228) - Add `node_id`, `pid`, attempt number for hanging tasks (#59793) - Revise resource allocator task scheduling to factor in pending task outputs (#60639) - Track block serialization time (#60574) - Use metrics from `OpRuntimeMetrics` for progress (#60304) - Tabular form for streaming executor op metrics (#59774) - Info-log cluster scale-up decisions (#60357) - Use plain mode instead of grid mode for `OpMetrics` logging (#59907) - Progress reporting refactors (#59350, #59629, #59880) - Remove deprecated `TENSOR_COLUMN_NAME` constant (#60573) - Remove `meta_provider` parameter (#60379) - Decouple Ray Train from Ray Data by removing top-level `ray.data` imports (#60292) - Move extension types to ray.data (#59420) - Skip upscaling validation warning for fixed-size actor pools (#60569) - Make `StatefulShuffleAggregation.finalize` allow incremental streaming (#59972) - Revisit `OutputSplitter` semantics to avoid unnecessary buffer accumulation (#60237) - Update to PyArrow 23 (#60739, #59489) - Add `BackpressurePolicy` to streaming executor progress bar (#59637) - Support Arrow-based transformations for preprocessors (#59810) - `StandardScaler` preprocessor with Arrow format (#59906) - OneHotEncoder with Arrow format (#59890) 🔨 Fixes - Fuse `MapBatches` even if they modify the row count (#60756) - Don't push limit past `map_batches` by default (#60448) - Fix wrong type hint of other dataset in zip and union (#60653) - Fix `ActorPoolMapOperator` to guarantee dispatch of all given inputs (#60763) - Fix `ArrowInvalid` error when backfilling missing fields from map tasks (#60643) - Fix attribute error in `UnionOperator.clear_internal_output_queue` (#60538) - Fix `DefaultClusterAutoscalerV2` raising KeyError: 'CPU' (#60208) - Fix `ReorderingBundleQueue` handling of empty output sequences (#60470) - Fix task completion time without backpressure grafana panel metric name (#60481) - Fix Union operator blocking when preserve _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.54.0]_ ### 2.53.0 — Ray-2.53.0 - Date: 2025-12-20 - Version: 2.53.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.53.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.53.0 - **added** — Add Kafka as a native datasource for data ingestion - **added** — Add Dataset.summary() API for quick dataset inspection - **added** — Add Iceberg schema evolution, upsert, and overwrite support - **added** — Add should_continue_on_error for graceful error handling in batch inference - **added** — Add datetime compute expressions support - **added** — Enable expressions for grouped with_column in Ray Data - **added** — Add new utilization-based cluster autoscaler for Ray Data workloads - **added** — Add deployment topology visibility in Serve REST API to visualize deployment dependency graphs - **added** — Add external_scaler_enabled flag to application config for third-party autoscaler integration - **added** — Add node rank and local rank support to replica rank system for multi-node deployments - **added** — Add custom batch size function support in @serve.batch - **added** — Add policy state persistence for custom autoscaling policies - **added** — Add Prometheus metrics for autoscaling decisions and router queue latency - **changed** — Parallelize DefaultCollateFn and arrow_batch_to_tensors - **changed** — Optimize autoscaler to support configurable step size for actor pool scaling - **changed** — Improve streaming repartition performance - **changed** — Add version support to read_lance() - **changed** — Add configurable batching for resolve_block_refs to speed up iter_batches - **fixed** — Fix support of list types for Unique aggregator - **fixed** — Fix reading from written parquet for numpy with NaNs - **removed** — Remove deprecated read_parquet_bulk API - **deprecated** — Ray plans to drop support for Pydantic V1 starting version 2.56.0 ##### Highlights * Ray plans to drop support for Pydantic V1 starting version 2.56.0. Please see [this RFC](https://github.com/ray-project/ray/issues/58876) for details. * Ray Data now has support for bounded reading from Kafka and improved Iceberg support. ##### Ray Data 🎉 New Features * **Autoscaling**: New utilization-based cluster autoscaler for Ray Data workloads (#59353, #59362, #59366). To use this new autoscaler set RAY\_DATA\_CLUSTER\_AUTOSCALER=V2. * **Kafka Datasource**: Add Kafka as a native datasource for data ingestion (#58592) * **Dataset summary API**: Add `Dataset.summary()` API for quick dataset inspection (#58862) * **Iceberg support**: Add Iceberg schema evolution, upsert, and overwrite support (#59210, #59335) * **Graceful error handling**: Add `should_continue_on_error` for graceful error handling in batch inference (#59212) * **Datetime compute expressions**: Add datetime compute expressions support (#58740) * **Grouped `with_column` expressions**: Enable expressions for grouped `with_column` in Ray Data (#58231) * **Parallelized collation**: Parallelize `DefaultCollateFn`, `arrow_batch_to_tensors` (#58821) 💫 Enhancements * **Optimized Autoscaler Step Size**: Optimize autoscaler to support configurable step size for actor pool scaling (#58726) * **Improved Streaming Repartition**: Improve streaming repartition performance (#58728) * **Actor init retry**: Add actor retry if there's a failure in `__init__` (#59105) * **Fused Repartition \+ MapBatches**: Fuse StreamingRepartition with MapBatches operators to scale collate (#59108) * **Combined repartitions**: Combine consecutive repartitions for efficiency (#59145) * **Prefetch buffering**: Handle prefetch buffering in `iter_batches` (#58657) * **HashShuffle block breakdown**: `HashShuffleAggregator` breaks down blocks on finalize (#58603) * **Backpressure tuning**: Tune concurrency cap backpressure object store budget ratio (#58813) * **Non-string ApproximateTopK**: Support non-string items for `ApproximateTopK` aggregator (#58659) * **Lance version support**: Add version support to `read_lance()` (#58895) * **Dashboard metrics**: Add `time_to_first_batch` and `get_ref_bundles` metrics to data dashboard (#58912) * **Iter prefetched bytes stats**: Add `iter_prefetched_bytes` statistics tracking (#58900) * **Configurable batching for `iter_batches`**: Add configurable batching for `resolve_block_refs` to speed up `iter_batches` (#58467) * **Improved dashboard metrics**: Improve Ray Data dashboard metrics display (#58667) * **Histogram percentiles**: Update Ray Data histograms to show percentiles in data dashboard (#58650) * **Deprecated API removal**: Remove deprecated `read_parquet_bulk` API (#58970) * **Block shaping option**: Add disable block shaping option to BlockOutputBuffer (#58757) * **Removed concurrency lock**: Remove concurrency lock for better performance (#56798) 🔨 Fixes * **Fixes to Unique**: Fix support of list types for Unique aggregator (#58916) * **Parquet NaN fix**: Fix reading from written parquet for numpy with NaNs (#59172) * **Hash Shuffle empty block**: Fix empty block sort in hash shuffle operator (#58836) * **Hive partitioning pushdown**: Fix pushdown optimizations with Hive partitioning (#58723) * **Object Store usage reporting**: Fix `obj_store_mem_max_pending_output_per_task` reporting (#58864) * **Pyarrow FileSystem serialization fix**: Handle filesystem serialization issue in `get_parquet_dataset` (#57047) * **Azure UC SAS**: Handle Azure UC user delegation SAS (#59393) * **Async UDF Thread Cleanup**: Close threads from async UDF after actor died (#59261) * **Object Locality Default**: Default return 0s for object locality instead of \-1s (#58754) 📖 Documentation * Added contributing guide to Ray Data documentation (#58589) * Added download expression to key user journeys in documentation (#59417) * Added Kafka user guide (#58881) * Added unstructured _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.53.0]_ ### 2.51.2 — Ray-2.51.2 - Date: 2025-11-29 - Version: 2.51.2 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.51.2 - Permalink: https://whatsnew.fyi/product/ray/releases/2.51.2 - **security** — reject Sec-Fetch-* other browser-specific headers in dashboard browser rejection logic for CVE-2025-62593 - Fix for CVE-2025-62593: reject Sec-Fetch-* other browser-specific headers in dashboard browser rejection logic ### 2.52.1 — Ray-2.52.1 - Date: 2025-11-28 - Version: 2.52.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.52.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.52.1 - **security** — Improved handling for CVE-2025-62593 by testing for more browser-specific headers in dashboard browser rejection logic - More robust handling for CVE-2025-62593: test for more browser-specific headers in dashboard browser rejection logic ### 2.52.0 — Ray-2.52.0 - Date: 2025-11-21 - Version: 2.52.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.52.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.52.0 - **removed** — End of Life for Python 3.9 Support: Ray will no longer be releasing Python 3.9 wheels from now on - **security** — Token authentication: Ray now supports built-in token authentication across all components including the dashboard, CLI, API clients, and internal services - **added** — Added predicate pushdown rule that pushes filter predicates past eligible operators - **added** — Iceberg support for upsert tables, schema updates, and overwrite operations - **added** — Iceberg support for predicate and projection pushdown - **added** — Iceberg write datafiles in write() then commit - **changed** — Enhanced Unity Catalog integration - **added** — Namespaced expressions that expose PyArrow functions - **added** — Added version argument to read_delta_lake - **added** — Generator UDF support for map_groups - **added** — ApproximateTopK aggregator - **added** — Serialization framework for preprocessors - **added** — Support for temporal, list, tensor, and struct datatypes - **changed** — Use approximate quantile for RobustScaler preprocessor - **changed** — Map batches support for limit pushdown - **changed** — Make all map operations zero-copy by default - **changed** — Use tqdm_ray for progress reporting from workers - **changed** — Improved concurrency cap backpressure tuning - **fixed** — Fixed renamed columns to be appropriately dropped from output - **fixed** — Fixed handling of renames in projection pushdown - **fixed** — Fixed broken LogicalOperator abstraction barrier in predicate pushdown rule #### Release Highlights **Ray Core:** - End of Life for Python 3.9 Support: Ray will no longer be releasing Python 3.9 wheels from now on. - Token authentication: Ray now supports built-in token authentication across all components including the dashboard, CLI, API clients, and internal services. This provides an additional layer of security for production deployments to reduce the risk of unauthorized code execution. Token authentication is initially off by default. For more information, see: https://docs.ray.io/en/latest/ray-security/token-auth.html **Ray Data:** - We’ve added a number of improvements for Iceberg, including upserts, predicate and projection pushdown, and overwrite. - We’ve added significant improvements to our expressions framework, including temporal, list, tensor, and struct datatype expressions. #### Ray Libraries ##### Ray Data 🎉 New Features: - Added predicate pushdown rule that pushes filter predicates past eligible operators ([#58150](https://github.com/ray-project/ray/pull/58150),[ #58555](https://github.com/ray-project/ray/pull/58555)) - Iceberg support for upsert tables, schema updates, and overwrite operations ([#58270](https://github.com/ray-project/ray/pull/58270)) - Iceberg support for predicate and projection pushdown ([#58286](https://github.com/ray-project/ray/pull/58286)) - Iceberg write datafiles in write() then commit ([#58601](https://github.com/ray-project/ray/pull/58601)) - Enhanced Unity Catalog integration ([#57954](https://github.com/ray-project/ray/pull/57954)) - Namespaced expressions that expose PyArrow functions ([#58465](https://github.com/ray-project/ray/pull/58465)) - Added version argument to read_delta_lake ([#54976](https://github.com/ray-project/ray/pull/54976)) - Generator UDF support for map_groups ([#58039](https://github.com/ray-project/ray/pull/58039)) - ApproximateTopK aggregator ([#57950](https://github.com/ray-project/ray/pull/57950)) - Serialization framework for preprocessors ([#58321](https://github.com/ray-project/ray/pull/58321)) - Support for temporal, list, tensor, and struct datatypes ([#58225](https://github.com/ray-project/ray/pull/58225)) 💫 Enhancements: - Use approximate quantile for RobustScaler preprocessor ([#58371](https://github.com/ray-project/ray/pull/58371)) - Map batches support for limit pushdown ([#57880](https://github.com/ray-project/ray/pull/57880)) - Make all map operations zero-copy by default ([#58285](https://github.com/ray-project/ray/pull/58285)) - Use tqdm_ray for progress reporting from workers ([#58277](https://github.com/ray-project/ray/pull/58277)) - Improved concurrency cap backpressure tuning ([#58163](https://github.com/ray-project/ray/pull/58163),[ #58023](https://github.com/ray-project/ray/pull/58023),[ #57996](https://github.com/ray-project/ray/pull/57996)) - Sample finalized partitions randomly to avoid lens effect ([#58456](https://github.com/ray-project/ray/pull/58456)) - Allow file extensions starting with '.' ([#58339](https://github.com/ray-project/ray/pull/58339)) - Set default file_extensions for read_parquet ([#56481](https://github.com/ray-project/ray/pull/56481)) - URL decode values in parse_hive_path ([#57625](https://github.com/ray-project/ray/pull/57625)) - Streaming partition enforces row_num per block ([#57984](https://github.com/ray-project/ray/pull/57984)) - Streaming repartition combines small blocks ([#58020](https://github.com/ray-project/ray/pull/58020)) - Lower DEFAULT_ACTOR_MAX_TASKS_IN_FLIGHT_TO_MAX_CONCURRENCY_FACTOR to 2 ([#58262](https://github.com/ray-project/ray/pull/58262)) - Set udf-modifying-row-count default to false ([#58264](https://github.com/ray-project/ray/pull/58264)) - Cache PyArrow schema operations ([#58583](https://github.com/ray-project/ray/pull/58583)) - Explain optimized plans ([#58074](https://github.com/ray-project/ray/pull/58074)) - Ranker interface ([#58513](https://github.com/ray-project/ray/pull/58513)) 🔨 Fixes: _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.52.0]_ ### 2.51.1 — Ray-2.51.1 - Date: 2025-11-01 - Version: 2.51.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.51.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.51.1 - **fixed** — Reuse previous metadata if transferring the same tensor list with nixl - Reuse previous metadata if transferring the same tensor list with `nixl` (https://github.com/ray-project/ray/pull/58309) ### 2.51.0 — Ray-2.51.0 - Date: 2025-10-29 - Version: 2.51.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.51.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.51.0 - **changed** — Ray Train v2 is now enabled by default, providing usability and stability improvements and new features - **added** — Application-level autoscaling in Ray Serve with custom autoscaling policies that operate across all deployments in an application - **added** — Enhanced autoscaling context with replica-level metrics including total_running_requests, total_queued_requests, and total_num_requests - **added** — Autoscaling metrics aggregation functions supporting min, max, and time-weighted average aggregation - **added** — Support for multiple task consumers in a single Ray Serve application - **added** — Enhanced support for Unity Catalog integration in Ray Data - **added** — New expression evaluator infrastructure for improved query optimization in Ray Data - **added** — Support for SaveMode in Ray Data write operations - **added** — Approximate quantile aggregator in Ray Data - **added** — MCAP datasource support for robotics data in Ray Data - **added** — Callback-based stat computation for preprocessors and ValueCounter in Ray Data - **added** — Support for multiple download URIs with improved error handling in Ray Data - **added** — Top-level ray.train aliases for public APIs - **changed** — Improved projection pushdown handling with renamed columns in Ray Data - **changed** — Enhanced hash-shuffle performance with better retry policies in Ray Data - **changed** — Reduced memory usage with improved streaming generator backpressure in Ray Data - **fixed** — Fixed renamed columns to be appropriately dropped from output in Ray Data - **fixed** — Fixed driver hang during streaming generator block metadata retrieval in Ray Data - **fixed** — Fixed ControllerError triggered by after_worker_group_poll_status errors in Ray Train - **fixed** — Fixed iter_torch_batches use of ray.train.torch.get_device outside Train #### Release Highlights **Ray Train:** - Ray Train v2 is now enabled by default! Ray Train v2 provides usability and stability improvements, as well as new features. For more details, see the [REP](https://github.com/ray-project/enhancements/blob/main/reps/2024-10-18-train-tune-api-revamp/2024-10-18-train-tune-api-revamp.md) and [Migration Guide](https://github.com/ray-project/ray/issues/49454). To disable Ray Train v2, set the environment variable `RAY_TRAIN_V2_ENABLED=0`. **Ray Serve:** - Application-level autoscaling: Introduces custom autoscaling policies that operate across all deployments in an application, enabling coordinated scaling decisions based on aggregate metrics. This is a significant advancement over per-deployment autoscaling, allowing for more intelligent resource management at the application level. - Enhanced autoscaling capabilities with replica-level metrics: Wires up `AutoscalingContext` with `total_running_requests`, `total_queued_requests`, and `total_num_requests`, plus adds support for min, max, and time-weighted average aggregation functions. These improvements give users fine-grained control to implement sophisticated custom autoscaling policies based on real-time workload metrics. #### Ray Libraries ##### Ray Data 🎉 New Features: - Added enhanced support for Unity Catalog integration (#57954, #58049) - New expression evaluator infrastructure for improved query optimization (#57778, #57855) - Support for SaveMode in write operations (#57946) - Added approximate quantile aggregator (#57598) - MCAP datasource support for robotics data (#55716) - Callback-based stat computation for preprocessors and ValueCounter (#56848) - Support for multiple download URIs with improved error handling (#57775) 💫 Enhancements: - Improved projection pushdown handling with renamed columns (#58033, #58037, #58040, #58071) - Enhanced hash-shuffle performance with better retry policies (#57572) - Streamlined concurrency parameter semantics (#57035) - Improved execution progress rendering (#56992) - Better handling of empty columns in pandas blocks (#57740) - Enhanced support for complex data types and column operations (#57271) - Reduced memory usage with improved streaming generator backpressure (#57688) - Enhanced preemption testing and utilities (#57883) - Improved Download operator display names (#57773) - Better handling of variable-shaped tensors and tensor columns (#57240) - Optimized aggregator execution with out-of-order processing by default (#57753) 🔨 Fixes: - Fixed renamed columns to be appropriately dropped from output (#58040, #58071) - Fixed handling of renames in projection pushdown (#58033, #58037) - Fixed vLLMEngineStage field name inconsistency for images (#57980) - Fixed driver hang during streaming generator block metadata retrieval (#56451) - Fixed retry policy for hash-shuffle tasks (#57572) - Fixed prefetch loop to avoid blocking on fetches (#57613) - Fixed empty projection handling (#57740) - Fixed errors with concatenation of mixed pyarrow native and extension types (#56811) 📖 Documentation: - Updated document embedding benchmark to use canonical Ray Data API (#57977) - Improved concurrency-related documentation (#57658) - Updated preprocessing and data handling examples ##### Ray Train 🎉 New features - Turn on Train v2 by default (#57857) - Top-level `ray.train` aliases for public APIs (#57758) 💫 Enhancements - Raise clear errors when mixing v1/v2 APIs (#57570) - JAX backend: add `jax.distributed.shutdown()` for `JaxBackend` (#57802) - Update `TrainingFailedError` module (#57865) - Improve deprecation handling when `ray.train` methods are called from `ray.tune` (#57810) - Enable deprecation warnings for legacy XGBoost/LightGBM trainers (#57280) 🔨 Fixes - Fix `ControllerError` triggered by `after_worker_group_poll_status` errors (#57869) - Fix `iter_torch_batches` use of `ray.train.torch.get_device` outsi _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.51.0]_ ### 2.50.1 — Ray-2.50.1 - Date: 2025-10-18 - Version: 2.50.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.50.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.50.1 - **fixed** — Fix deadlock when cancelling stale requests on in-order actors Ray Core: Fix deadlock when cancelling stale requests on in-order actors (#57746) ### 2.50.0 — Ray-2.50.0 - Date: 2025-10-10 - Version: 2.50.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.50.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.50.0 - **changed** — Default shuffle strategy changed from sort-based to hash-based for improved shuffle performance and lower peak memory usage for aggregations - **added** — New expression API enables predicate-based filtering, UDF transformations with with_column, and column aliasing for more powerful data transformations - **added** — Added support for projection pushdown into Parquet reads - **added** — New download expression enables efficient loading of data from columns containing URIs with improved performance and error handling - **added** — New explain() API provides insights into dataset execution plans - **added** — Added streaming_train_test_split to avoid materialization for train/test splits - **added** — Enabled multi-node tensor and pipeline parallelism for Ray Data LLM processing - **added** — Added chat_template_kwargs parameter for customizing chat templates in Ray Data LLM - **added** — Added support for OpenAI's nested image URL format in Ray Data LLM multimodal pipelines - **added** — vLLM engines can now be shared across sequential processors for better resource utilization - **added** — Enhanced Dataset.stats() output with input/output row counts per operator - **added** — Added new metrics for task duration, inputs per task, and output blocks - **added** — Added time to first batch metric for better iteration performance monitoring - **added** — Added type-specific aggregators for numerical, categorical, and vector columns - **added** — Added fine-grained concurrency controls with max_task_concurrency and resource allocation options - **added** — Alpha release of Ray Direct Transport enabling GPU data to remain in GPU memory with efficient data transfer via collective communication and RDMA - **added** — Added StepFailedRecreateEnv exception for users with unsatisfiable environments in RLLib - **added** — Async Inference alpha release for Ray Serve with APIs supporting long-running asynchronous inference tasks and message broker support - **added** — Added support for replica ranks in Ray Serve for large-model inference use-cases - **added** — Added FastAPI factory pattern support in Ray Serve for using non-serializable FastAPI plugins - **added** — zip() operator now supports combining multiple datasets, not just pairs - **changed** — Concurrency parameter now accepts tuples for more flexible configuration - **changed** — Resource requirements (num_cpus/gpus, memory) are now top-level parameters in most APIs for easier configuration - **changed** — Write operations now use iterators instead of accumulating blocks in memory - **changed** — Improved groupby performance with sort-shuffle pull-based approach - **changed** — Improved join operations with new abstractions - **changed** — Improved compatibility between PyArrow native types, extension types, and pandas Arrow dtypes - **changed** — Joins now supported with list/tensor non-key columns - **changed** — Enhanced support for variable-shaped tensor arrays with different dimensions - **changed** — Added serialization/deserialization for PyArrow Extension Arrays - **changed** — Reduced memory usage for OneHotEncoder - **changed** — Reduced memory usage for schema unification - **changed** — Eliminated unnecessary block copying and double execution of arrow conversions - **changed** — Improved Parquet encoding ratio estimation - **changed** — Enabled per-block limiting for Limit operator - **changed** — Optimized schema handling with deduplication and removed unnecessary unification - **fixed** — Fixed streaming executor to properly drain output queues - **fixed** — Fixed retry logic for hash shuffle operations - **fixed** — Fixed split_blocks to prevent producing empty blocks #### Release Highlights **Ray Data:** This release offers many updates to Ray Data, including: - The default shuffle strategy is now changed from sort-based to hash-based. This will result in much lower peak memory usage and improved shuffle performance for aggregations. - We’ve added a new expression API enables predicate-based filtering, UDF transformations with `with_column`, and column aliasing for more powerful data transformations - Ray Data LLM has a number of new enhancements for multimodal data pipelines, including multi-node tensor and pipeline parallelism support per replica and ability to share vLLM engines across processors. **Ray Core:** Alpha release of **Ray Direct Transport** (formerly GPU objects) - simply enable it by adding the `tensor_transport` parameter to the existing native Ray Core API. This keeps GPU data in GPU memory until a transfer is needed, avoiding expensive serialization and copies to and from the Ray object store. It uses efficient data transports such as collective communication libraries (GLOO or NCCL) or point-to-point RDMA (via NVIDIA’s NIXL) to transfer data directly between devices, including both CPUs and GPUs. **Ray Train:** Local mode support for multi-process training with `torchrun`, enhanced checkpoint management with new upload modes and validation functions **Ray Serve:** - Async Inference alpha release - New Ray Serve APIs for supporting long-running asynchronous inference tasks, such as for video or large document processing. Includes capabilities for using different message brokers, adapters like `celery` and DLQ. - Support for replica ranks - Replica level ranks are added for supporting large-model inference use-cases such as wide Data Parallel and Expert Parallel setups. - FastAPI factory pattern support - Enables using FastAPI plugins that are not serializable via cloudpickle. - Throughput optimizations - Enable these using the `RAY_SERVE_THROUGHPUT_OPTIMIZED` environment variable. **RLLib:** Add `StepFailedRecreateEnv` exception for users with unsatisfiable environments **Ray Serve/Data LLM:** Improvements to multi node serving, loading models from remote storages, and sharing resources for efficiency (fractional gpus, sharing gpus on a data pipeline with shared stages) #### Ray Libraries ##### Ray Data 🎉 New Features: - **Expression and Filtering API:** New expression API enables predicate-based filtering, UDF transformations with with_column, and column aliasing for more powerful data transformations (#56716, #56313, #56550, #55915, #55788, #56193, #56596) - Added support for projection pushdown into Parquet reads (#56500) - New download expression enables efficient loading of data from columns containing URIs with improved performance and error handling (#55824, #56462, #56294, #56852, #57146) - New `explain()` API provides insights into dataset execution plans (#55482) - Added `streaming_train_test_split` to avoid materialization for train/test splits (#56803) - **Ray Data LLM:** - Enabled multi-node tensor and pipeline parallelism for LLM processing (#56779) - Added `chat_template_kwargs` parameter for customizing chat templates (#56490) - Added support for OpenAI's nested image URL format in multimodal pipelines (#56584) - vLLM engines can now be shared across sequential processors for better resource utilization (#55179) - Enhanced Dataset.stats() output with input/output row counts per operator (#56040) - Added new metrics for task duration, inputs per task, and output blocks (#56958, #56379) - Time to first batch metric for better iteration performance monitoring (#55758) - Added type-specific aggregators for numerical, categorical, and vector columns (#56610) - Added fine-grained concurrency controls with `max_task_concurrency` and resource allocation options (#56370, #56381) 💫 Enhancements: - **Join and shuffle improvements:** - Default shuffle strategy changed from sort-b _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.50.0]_ ### 2.49.2 — Ray-2.49.2 - Date: 2025-09-19 - Version: 2.49.2 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.49.2 - Permalink: https://whatsnew.fyi/product/ray/releases/2.49.2 There is no difference between 2.49.2 and 2.49.1, though we needed a patch version for other out of band reasons. To fill the awkward blankness, here is a haiku about Ray: Summit drawing near Ray advances, step by step Scaling without end ### 2.49.1 — Ray-2.49.1 - Date: 2025-09-03 - Version: 2.49.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.49.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.49.1 - **fixed** — Fix issue where GPU metrics are missing in Ray Dashboard - **fixed** — Fix regression in handling very large schemas in Ray Data - Ray Dashboard: Fix issue where GPU metrics are missing (#56006) - Ray Data: Fixed regression in handling very large schemas (#56058) ### 2.49.0 — Ray-2.49.0 - Date: 2025-08-26 - Version: 2.49.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.49.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.49.0 - **added** — Wrapped batch indices in a BatchMetadata object to make per-batch metadata explicit - **added** — Added support for Anti/Semi Join types - **added** — Introduced an Issue Detection Framework - **added** — Added an option to enable out-of-order execution for better performance - **added** — Introduced a StreamingSplit logical operator for DAG rewrite - **added** — Added a stratify parameter to train_test_split - **added** — Added Snowflake connectors - **added** — Updated Hudi integration to support incremental query - **added** — Added an Actor location tracker - **added** — Added BundleQueue.has_next - **added** — Made DEFAULT_OBJECT_STORE_MEMORY_LIMIT_FRACTION configurable - **added** — Added Expression support and a with_columns API - **added** — Allocate GPU resources in ResourceManager - **added** — Introduced JaxTrainer with SPMD support on TPUs - **changed** — Decoupled actor and node autoscaling; autoscaling now also considers budget - **changed** — Faster hash-shuffle resource usage calculation and more accurate shuffle progress totals - **changed** — Reduced Parquet metadata storage usage - **changed** — map_batches now defaults to row_modification=False and avoids pushing past limit - **changed** — ray.train.get_dataset_shard now lazily configures dataset sharding for better startup behavior - **fixed** — AutoscalingActorPool now properly downscales after execution #### Release Highlights **Ray Data:** - We’ve implemented a variety of performance enhancements, including improved actor/node autoscaling with budget-aware decisions; faster/more accurate shuffle accounting; reduced Parquet metadata footprint; and out-of-order execution for higher throughput. - We’ve also implemented anti/semi joins, stratified train_test_split, and added Snowflake connectors. **Ray Core:** - Performance/robustness cleanups around GCS publish path and raylet internals; simpler OpenTelemetry flagging; new user-facing API to wait for GPU tensor free; plus assorted test/infra tidy-ups **Ray Train:** - We’ve introduced a new JaxTrainer with SPMD support for TPUs. **Ray Serve:** - Custom Autoscaling per Deployment Serve now supports user-defined autoscaling policies via AutoscalingContext and AutoscalingPolicy, enabling fine-grained scaling logic at the deployment level. This is part of a large effort where we are adding support for autoscaling based on custom metrics in Serve, see [this RFC](https://github.com/ray-project/ray/issues/41135#issuecomment-3156717488) for more details. - Async Inference (Initial Support): Ray Serve introduces asynchronous inference execution, laying the foundation for better throughput and latency in async workloads. Please see [this RFC](https://github.com/ray-project/ray/issues/54652) for more details. - Major Performance Gains: This version of ray serve brings double digit % performance improvements both in throughput and latency. See release notes for more details. **Ray Serve/Data LLM:** - We’ve refactored Ray Serve LLM to be fully compatible with the default vllm serve and also now supports vLLM=0.10. - We’ve added a prefix cache-aware router with PrefixCacheAffinityRouter for optimized cache utilization; dynamic cache management via reset prefix cache remote methods; enhanced LMCacheConnectorV1 with kv_transfer_config support. #### Ray Libraries ##### Ray Data 🎉 New Features: - Wrapped batch indices in a BatchMetadata object to make per-batch metadata explicit. (#55643) - Added support for Anti/Semi Join types. (#55272) - Introduced an Issue Detection Framework. (#55155) - Added an option to enable out-of-order execution for better performance. (#54504) - Introduced a StreamingSplit logical operator for DAG rewrite. (#54994) - Added a stratify parameter to train_test_split. (#54624) - Added Snowflake connectors. (#51429) - Updated Hudi integration to support incremental query. (#54301) - Added an Actor location tracker. (#54590) - Added BundleQueue.has_next. (#54710) - Made DEFAULT_OBJECT_STORE_MEMORY_LIMIT_FRACTION configurable. (#54873) - Added Expression support & a with_columns API. (#54322) - Allocate GPU resources in ResourceManager. (#54445) 💫 Enhancements: - Decoupled actor and node autoscaling; autoscaling now also considers budget. (#55673, #54902) - Faster hash-shuffle resource usage calculation; more accurate shuffle progress totals. (#55503, #55543) - Reduced Parquet metadata storage usage. (#54821) - Export API improvements: refresh dataset/operator state, sanitize metadata, and truncate exported metadata. (#55355, #55379, #55216, #54623) - Metrics & observability: task metric improvements, external-buffer block-count metric, row-based metrics, clearer operator names in logs, single debug log when aggregators are ready. (#55429, #55022, #54693, #52949, #54483) - Dashboard: added “Max Bytes to Read” panel/budget, panels for blocks-per-task and bytes-per-block, and streaming executor duration. (#55024, #55020, #54614) - Planner/execution & infra cleanups: ExecutionResources and StatsManager cleanup, planner interface refactor, node trackers init, removed ray.get in _MapWorker ctor, removed target_shuffle_max_block_size. (#54694, #55400, #55018, #54665, #54734, #55158) - Behavior/interop tweaks: map_batches defaults to row_modification=False and avoids pushing past limit; limited operator pushdown; _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.49.0]_ ### 2.48.0 — Ray-2.48.0 - Date: 2025-07-18 - Version: 2.48.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.48.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.48.0 - **added** — Add reading from Delta Lake tables and Unity Catalog integration - **added** — Enhanced pin_memory support in iter_torch_batches - **added** — Added broadcast_from_rank_zero and barrier collective operations - **added** — Enhanced hardware metrics integration with Grafana dashboards - **added** — Added support for dynamically loading callbacks via environment variables - **added** — Added RouterConfig field to DeploymentConfig for custom RequestRouter configuration - **added** — Added support for implementing custom request routing algorithms - **added** — Support separate deployment config for PDProxy in PrefixAwareReplicaSet - **added** — Support for prefix-aware request router - **changed** — Enhanced handling of mismatched columns and pandas.NA values - **changed** — Improved read_text trailing newline semantics - **changed** — Optimized backpressure handling with policy-based resource management - **changed** — Enhanced write_parquet with support for both partition_by and row limits - **changed** — Improved execution performance for One Hot encoding in preprocessors - **changed** — Improved checkpoint population from before_init_train_context - **changed** — Enhanced controller state logging and metrics - **changed** — Enhanced FastAPI ingress deployment validation for multiple deployments - **changed** — Optimized get_live_deployments performance - **fixed** — Fixed map_groups issues - **fixed** — Prevented Op fusion for streaming repartition to avoid performance degradation - **fixed** — Fixed ActorPool autoscaler scaling up logic - **fixed** — Resolved empty dataset repartitioning issues #### Release Highlights - **Ray Data**: This release features a new Delta Lake and Unity Catalog integration and performance improvements to various reading/writing operators. - **Ray Core**: Enhanced GPU object support with intra-process communication and improved Autoscaler v2 functionality - **Ray Train**: Improved hardware metrics integration with Grafana and enhanced collective operations support - **Ray Serve LLM**: This release features early proof of concept for prefill-decode disaggregation deployment and LLM-aware request routing such as prefix-cache aware routing. - **Ray Data LLM**: Improved throughput and CPU memory utilization for ray data workers. #### Ray Libraries ##### Ray Data 🎉 New Features: - Add reading from Delta Lake tables and Unity Catalog integration (#53701) - Enhanced pin_memory support in iter_torch_batches (#53792) - Add pin_memory to iter_torch_batches (#53792) 💫 Enhancements: - Re-enabled sorting in Ray Data tests with performance improvements (#54475) - Enhanced handling of mismatched columns and pandas.NA values (#53861, #53859) - Improved read_text trailing newline semantics (#53860) - Optimized backpressure handling with policy-based resource management (#54376) - Enhanced write_parquet with support for both partition_by and row limits (#53930) - Prevent filename collisions on write operations (#53890) - Improved execution performance for One Hot encoding in preprocessors (#54022) 🔨 Fixes: - Fixing map_groups issues (#54462) - Prevented Op fusion for streaming repartition to avoid performance degradation (#54469) - Fixed ActorPool autoscaler scaling up logic (#53983) - Resolved empty dataset repartitioning issues (#54107) - Fixed PyArrow overflow handling in data processing (#53971, #54390) - Fixed IcebergDatasink to properly generate individual file uuids (#52956) - Avoid OOMs with read_json(..., lines=True) (#54436) - Handle HuggingFace parquet dataset resolve URLs (#54146) - Fixed BlockMetadata derivation for Read operator (#53908) 📖 Documentation: - Updated AggregateFnV2 documentation to clarify finalize method (#53835) - Improved preprocessor and vectorizer API documentation ##### Ray Train 🎉 New Features: - Added broadcast_from_rank_zero and barrier collective operations (#54066) - Enhanced hardware metrics integration with Grafana dashboards (#53218) - Added support for dynamically loading callbacks via environment variables (#54233) 💫 Enhancements: - Improved checkpoint population from before_init_train_context (#54453) - Enhanced controller state logging and metrics (#52805) - Added structured logging environment variable support (#52952) - Improved handling of Noop scaling decisions for smoother scaling logic (#53180) - Logging of controller state transitions to aid in debugging and analysis (#53344) 🔨 Fixes: - Fixed GPU tensor reporting in ray.train.report (#53725) - Enhanced move_tensors_to_device utility for complex tensor structures (#53109) - Improved worker health check error handling with trace information (#53626) - Fixed GPU transfer support for non-contiguous tensors (#52548) - Force abort on SIGINT spam and do not abort finished runs (#54188) 📖 Documentation: - Updated beginner PyTorch example (#54124) - Added documentation for ray.train.collective APIs (#54340) - Added a note about PyTorch DataLoader's multiprocessing and forkserver usage (#52924) - Fixed various docstring format and indentation issues (#52855, #52878) - Added note that ray.train.report API docs should mention optional checkpoint_dir_name (#54391) 🏗 Architecture refactoring: - Removed subclass relationship between RunConfig and RunConfigV1 (#54293) - Enhanced error handling for finished training runs (#54188) - Deduplicated ML doctest runners in CI for efficiency (#53157) - Converted isort configuration to Ruff for consistency (#52869) ##### Ray Tune 💫 Enhancements: - Updated tes _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.48.0]_ ### 2.47.1 — Ray-2.47.1 - Date: 2025-06-18 - Version: 2.47.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.47.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.47.1 - **fixed** — Ray failed to start on Mac Ray 2.47.1 fixed an issue where Ray failed to start on Mac (https://github.com/ray-project/ray/pull/53807) ### 2.47.0 — Ray-2.47.0 - Date: 2025-06-12 - Version: 2.47.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.47.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.47.0 - **added** — Support prefill disaggregation in Ray Serve LLM for production LLM serving use cases - **added** — Add save modes support to file data sinks - **added** — Add flattening capability to the Concatenator preprocessor to support output vectorization use cases - **added** — Support for configuring Ray Train worker actor runtime environments - **added** — Include Grafana panel data in Ray Train export for improved monitoring - **added** — Introduce a structured logging environment variable to standardize log formats - **added** — Add metrics for TrainControllerState to enhance observability - **added** — Add support for implementing custom request routing algorithms in Ray Serve - **added** — Introduce an environment variable to prioritize custom resources during deployment scheduling - **added** — Expose vLLM Metrics to serve.llm API - **added** — Add Embedding API to Ray Serve/Data LLM - **added** — Add GPU inference to offline evaluation in RLlib - **changed** — Re-enable Actor locality-based scheduling with improved algorithms for ranking bundle locations - **changed** — Disable blocking pipeline by default until Actor Pool fully scales up to min actors - **changed** — Improve progress bar and dashboard to show name of partial functions properly - **changed** — The ingress API now accepts a builder function in addition to an ASGI app object - **changed** — Allow setting name_prefix in build_llm_deployment - **changed** — Mitigate the serve.llm streaming overhead by properly batching stream chunks - **fixed** — Make Ray Data from_torch respect Dataset len - **fixed** — Fix race condition bug in fault tolerance by disabling on_exit hook - **fixed** — Fix move_tensors_to_device utility for the list/tuple[tensor] case - **fixed** — Fix ActorPool scaling to avoid scaling down when the input queue is empty - **fixed** — Fix internal queues accounting for all Operators with an internal queue - **fixed** — Fix backpressure for FileBasedDatasource to prevent potential OOMs for workloads using FileBasedDatasources - **fixed** — Fixed GPU transfer support for non-contiguous tensors - **fixed** — Fixed runtime_env validation for py_modules - **fixed** — Disallow special characters in Serve deployment and application names - **fixed** — Add a descriptive error message when a deployment name is not found - **fixed** — LLMRouter.check_health() should check LLMServer.check_health() - **fixed** — Fix runtime passthrough and auto-executor class selection - **fixed** — Update check_health return type - **fixed** — Bug fix for duplication of token - **fixed** — In stream batching, first part of the stream was always consumed and not streamed back from the router - **fixed** — Fixed weight synching in offline evaluation - **fixed** — Fixed bug in split_and_zero_pad utility function related to complex structures vs simple values or np.arrays #### Release Highlights * Prefill disaggregation is now supported in initial support in Ray Serve LLM (#53092). This is critical for production LLM serving use cases. * Ray Data features a variety of performance improvements (locality-based scheduling, non-blocking execution) as well as improvements to observability, preprocessors, and other stability fixes. * Ray Serve now features custom request routing algorithms, which is critical for high throughput traffic for large model use cases. #### Ray Libraries ##### Ray Data 🎉 New Features: - Add save modes support to file data sinks (#52900) - Added flattening capability to the Concatenator preprocessor to support output vectorization use cases (#53378) 💫 Enhancements: - Re-enable Actor locality-based scheduling. This PR also improves algorithms for ranking the locations for the bundle. (#52861) - Disable blocking pipeline by default until Actor Pool fully scales up to min actors (#52754) - Progress bar and dashboard improvements to show name of partial functions properly(#52280) 🔨 Fixes: - Make Ray Data `from_torch` respect Dataset len (#52804) - Fixing flaky aggregation test (#53383) - Fix race condition bug in fault tolerance by disabling `on_exit` hook (#53249) - Fix `move_tensors_to_device` utility for the list/tuple[tensor] case (#53109) - Fix `ActorPool` scaling to avoid scaling down when the input queue is empty (#53009) - Fix internal queues accounting for all Operators w/ an internal queue (#52806) - Fix backpressure for `FileBasedDatasource`. This fixes potential OOMs for workloads using `FileBasedDatasources` (#52852) 📖 Documentation: - Fix working code snippets (#52748) - Improve AggregateFnV2 docstrings and examples (#52911) - Improved documentation for vectorizers and API visibility in Data (#52456) ##### Ray Train 🎉 New Features: - Added support for configuring Ray Train worker actor runtime environments. (#52421) - Included Grafana panel data in Ray Train export for improved monitoring. (#53072) - Introduced a structured logging environment variable to standardize log formats. (#52952) - Added metrics for `TrainControllerState` to enhance observability. (#52805) 💫 Enhancements: - Logging of controller state transitions to aid in debugging and analysis. (#53344) - Improved handling of `Noop` scaling decisions for smoother scaling logic. (#53180) 🔨 Fixes: - Improved `move_tensors_to_device utility` to correctly handle `list` / `tuple` of tensors. (#53109) - Fixed GPU transfer support for non-contiguous tensors. (#52548) - Increased timeout in `test_torch_device_manager` to reduce flakiness. (#52917) 📖 Documentation: - Added a note about PyTorch DataLoader’s multiprocessing and forkserver usage. (#52924) - Fixed various docstring format and indentation issues. (#52855, #52878) - Removed unused "configuration-overview" documentation page. (#52912) - General typo corrections. (#53048) 🏗 Architecture refactoring: - Deduplicated ML doctest runners in CI for efficiency. (#53157) - Converted isort configuration to Ruff for consistency. (#52869) - Removed unused `PARALLEL_CI` blocks and combined imports. (#53087, #52742) ##### Ray Tune 💫 Enhancements: - Updated `test_train_v2_integration` to use the correct `RunConfig`. (#52882) 📖 Documentation: - Replaced `session.report` with `tune.report` and corrected import paths. (#52801) - Removed outdated graphics cards reference in docs. (#52922) - Fixed various docstring format issues. (#52879) ##### Ray Serve 🎉 New Features: - Added support for implementing custom request routing algorithms. (#53251) - Introduced an environment variable to prioritize custom resources during deployment scheduling. (#51978) 💫 Enhancements: - The ingress API now accepts a builder function in addition to an ASGI app object. (#52892) 🔨 Fixes: - Fixed `runtime_env` validation for `py_modules`. (#53186) - Disallowed special character _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.47.0]_ ### 2.46.0 — Ray-2.46.0 - Date: 2025-05-07 - Version: 2.46.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.46.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.46.0 - **added** — Ray Data now supports hash-shuffle based repartitioning and aggregations - **added** — Ray Data now supports joins using hash-shuffle - **added** — Ray Serve LLM is now compatible with vLLM v1 - **added** — Add initial Train Grafana dashboard with built-in metrics for training workloads - **added** — Offline Evaluation with loss function for Offline RL pipeline, introducing three new callbacks: on_offline_evaluate_start, on_offline_evaluate_end, on_offline_eval_runners_recreated - **added** — New custom_data attribute for SingleAgentEpisode and MultiAgentEpisode to store custom metrics - **changed** — Add memory attribute to ExecutionResources - **changed** — Support ray_remote_args for read_tfrecords - **changed** — Add PhysicalOperator.min_max_resource_usage_bounds - **changed** — Support num_cpus, memory, concurrency, batch_size for preprocess - **changed** — Support new pyiceberg version - **changed** — Lazily import torch FSDP for ray.train.torch module to improve performance and reduce unnecessary dependencies - **changed** — Deserialize the user-defined training function directly on workers, improving efficiency - **changed** — Refactor LLMServer and LLMEngine to not diverge too much from vllm chat formatting logic - **changed** — Add router replicas and batch size to llm config - **changed** — Bump vllm from 0.8.2 to 0.8.5 - **fixed** — Handle Arrow Array null types in to_numpy - **fixed** — Fix S3 serialization wrapper compatibility with RetryingPyFileSystem - **fixed** — Fix Optimizer to apply rules until plan stabilizes - **fixed** — Fix FuseOperators rule to properly handle transformations that drastically change size of the dataset - **fixed** — Fix error when no arguments are passed into TorchTrainer #### Release Highlights The 2.46 Ray release comes with a couple core highlights: 1. Ray Data now supports **hash shuffling for repartition and aggregations, along with support for joins**. This enables many new data processing workloads to be run on Ray Data. Please give it a try and let us know if you have any feedback! 2. Ray Serve LLM now **supports vLLM v1** to be forward-compatible with upcoming vLLM releases. This also opens up significant performance improvements that come with vLLM's v1 refactor. 3. There is a new **Train Grafana dashboard** which provides in-depth metrics on Grafana for better metrics on training workloads. #### Ray Libraries ##### Ray Data 🎉 New Features: - Adding support for hash-shuffle based repartitioning and aggregations (#52664) - Added support for Joins (using hash-shuffle) (#52728) - [LLM] vLLM support upgrades to 0.8.5 (#52344) 💫 Enhancements: - Add memory attribute to ExecutionResources (#51127) - Support ray_remote_args for read_tfrecords #52450 - [data.dashboard] Skip reporting internal metrics (#52666) - Add PhysicalOperator.min_max_resource_usage_bounds (#52502) - Speed up printing the schema (#52612) - [data.dashboard] Dataset logger for worker (#52706) - Support new pyiceberg version (#51744) - Support num_cpus, memory, concurrency, batch_size for preprocess (#52574) 🔨 Fixes: - Handle Arrow Array null types in to_numpy (#52572) - Fix S3 serialization wrapper compatibility with RetryingPyFileSystem (#52568) - Fixing Optimizer to apply rules until plan stabilize; (#52663) - Fixing FuseOperators rule to properly handle the case of transformations drastically changing size of the dataset (#52570) 📖 Documentation: - [LLM] Improve concurrency settings, improve prompt to achieve better throughput (#52634) ##### Ray Train 🎉 New Features: - Add initial Train Grafana dashboard (#52709) 💫 Enhancements: - Lazily import torch FSDP for ray.train.torch module to improve performance and reduce unnecessary dependencies (#52707) - Deserialize the user-defined training function directly on workers, improving efficiency (#52684) 🔨 Fixes: - Fixed error when no arguments are passed into TorchTrainer (#52693) 📖 Documentation: - Added new XGBoostTrainer user guide (#52355) 🏗 Architecture refactoring: - Re-enabled isort for python/ray/train to maintain code formatting consistency (#52717) ##### Ray Tune 📖 Documentation: - Fixed typo in Ray Tune PyTorch Lightning docs (#52756) ##### Ray Serve 💫 Enhancements: - [LLM] Refactor LLMServer and LLMEngine to not diverge too much from vllm chat formatting logic ([#52597](https://github.com/ray-project/ray/pull/52597)) - Bump vllm from 0.8.2 to 0.8.5 in /python ([#52344](https://github.com/ray-project/ray/pull/52344)) - [LLM] Add router replicas and batch size to llm config ([#52655](https://github.com/ray-project/ray/pull/52655)) 🔨 Fixes: - Request cancellation not propagating correctly across deployments (#52591) - BackpressureError not properly propagated in FastAPI ingress deployments (#52397) - Hanging issue when awaiting deployment responses (#52561) - [Serve.llm] made Ray Serve LLM compatible with vLLM v1 ([#52668](https://github.com/ray-project/ray/pull/52668)) 📖 Documentation: - [Serve][LLM] Add doc for deploying DeepSeek ([#52592](https://github.com/ray-project/ray/pull/52592)) ##### RLLib 🎉 New Features: - Offline Evaluation with loss function for Offline RL pipeline. Introduces three new callbacks, `on_offline_evaluate_start`, `on_offline_evaluate_end`, `on_offline_eval_runners_recreated` (#52308) 💫 Enhancements: - New `custom_data` attribute for `SingleAgentEpisode` and `MultiAgentEpisode` to store custom metrics. Deprecates `add|get_temporary_timestep_data()` (#52603) #### Ray Core 💫 Enhancements: - Only get serialization context once for all .remote args (#52690) - Add grpc server success and fail count metric (#52711) 🔨 Fixes: - Fix op _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.46.0]_ ### 2.45.0 — Ray-2.45.0 - Date: 2025-04-29 - Version: 2.45.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.45.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.45.0 - **changed** — Make Object Store Fallback Directory configurable - **added** — Support with_tensor_transport(transport='shm') in cgraph - **added** — Support reduce scatter and all gather collective for GPU communicator in compiled graph - **fixed** — Make sure KillActor RPC with force_kill=True can actually kill the threaded actor - **fixed** — Do not remove idle nodes for upcoming placement groups in Autoscaler - **fixed** — Prevent threaded actors from getting stuck forever when receiving two exit signals - **fixed** — Fix illegal memory access of cgraph when used in PP - **fixed** — Avoid resubmitted actor tasks from hanging indefinitely - **fixed** — Fix interleaved placement group creation process due to node failure - **fixed** — Flush task events in CoreWorker::Shutdown instead of CoreWorker::Disconnect - **changed** — Split dashboard single process into multiple processes to improve stability and avoid interference between different heads - **added** — Add ClickHouse sink via Dataset.write_clickhouse() - **added** — Support ray_remote_args_fn in Dataset.groupby().map_groups() to set per-group runtime env and resource hints - **added** — Expose Dataset.name and set_name as public API for easier lineage tracking - **added** — Allow async callable classes in Dataset.flat_map() - **added** — Introduce Ruleset abstraction for rule-based query optimisation - **added** — Add seamless conversion from Daft DataFrame to Ray Dataset - **added** — Improve support for line-delimited JSONL reading in read_json() - **added** — Provide Dataset.export_metadata() for schema and stats snapshots - **changed** — Improve performance of sorting and sort-shuffle based operations by more than 5x #### Ray Core 💫 Enhancements - Make Object Store Fallback Directory configurable ([#51189](https://github.com/ray-project/ray/pull/51189)). - [cgraph] Support `with_tensor_transport(transport='shm')` ([#51872](https://github.com/ray-project/ray/pull/51872)). - [cgraph] Support reduce scatter and all gather collective for GPU communicator in compiled graph ([#50624](https://github.com/ray-project/ray/pull/50624)). 🔨 Fixes - Make sure `KillActor` RPC with `force_kill=True` can actually kill the threaded actor ([#51414](https://github.com/ray-project/ray/pull/51414)). - [Autoscaler] Do not remove idle nodes for upcoming placement groups ([#51122](https://github.com/ray-project/ray/pull/51122)). - Threaded actors get stuck forever if they receive two exit signals ([#51582](https://github.com/ray-project/ray/pull/51582)). - [cgraph] Fix illegal memory access of cgraph when used in PP ([#51734](https://github.com/ray-project/ray/pull/51734)). - Avoid resubmitted actor tasks from hanging indefinitely ([#51904](https://github.com/ray-project/ray/pull/51904)). - Fix interleaved placement group creation process due to node failure ([#52202](https://github.com/ray-project/ray/pull/52202)). - Flush task events in `CoreWorker::Shutdown` instead of `CoreWorker::Disconnect` ([#52374](https://github.com/ray-project/ray/pull/52374)). 🏗 Architecture refactoring - Split dashboard single process into multiple processes to improve stability and avoid interference between different heads ([#51282](https://github.com/ray-project/ray/pull/51282), [#51489](https://github.com/ray-project/ray/pull/51489), [#51555](https://github.com/ray-project/ray/pull/51555), [#51507](https://github.com/ray-project/ray/pull/51507), [#51587](https://github.com/ray-project/ray/pull/51587), [#51553](https://github.com/ray-project/ray/pull/51553), [#51676](https://github.com/ray-project/ray/pull/51676), [#51733](https://github.com/ray-project/ray/pull/51733), [#51809](https://github.com/ray-project/ray/pull/51809), [#51877](https://github.com/ray-project/ray/pull/51877), [#51876](https://github.com/ray-project/ray/pull/51876), [#51980](https://github.com/ray-project/ray/pull/51980), [#52114](https://github.com/ray-project/ray/pull/52114)). #### Ray Libraries ##### Ray Data 🎉 New Features - **New ClickHouse sink** via `Dataset.write_clickhouse()` ([#50377](https://github.com/ray-project/ray/pull/50377)) - Support `ray_remote_args_fn` in `Dataset.groupby().map_groups()` to set per-group runtime env and resource hints ([#51236](https://github.com/ray-project/ray/pull/51236)) - Expose `Dataset.name` / `set_name` as a public API for easier lineage tracking ([#51076](https://github.com/ray-project/ray/pull/51076)) - Allow async callable classes in `Dataset.flat_map()` ([#51180](https://github.com/ray-project/ray/pull/51180)) - Introduce **Ruleset** abstraction for rule-based query optimisation ([#51558](https://github.com/ray-project/ray/pull/51558)) - Add seamless conversion from Daft DataFrame to Ray Dataset ([#51531](https://github.com/ray-project/ray/pull/51531)) - Improved support for line-delimited JSONL reading in `read_json()` ([#52083](https://github.com/ray-project/ray/pull/52083)) - Provide `Dataset.export_metadata()` for schema & stats snapshots ([#52227](https://github.com/ray-project/ray/pull/52227)) 💫 Enhancements - **Improved performance of sorting and sort-shuffle based operations** (by more than 5x in benchmarks) ([#51943](https://github.com/ray-project/ray/pull/51943/files)) - Metrics: number of map-actor workers **alive / pending / restarting** ([#51082](https://github.com/ray-project/ray/pull/51082)) - Continuous memory-usage polling per map task ([#51324](https://github.com/ray-project/ray/pull/51324)) - Auto-tune map-task memory based on output size ([#51536](https://github.com/ray-project/ray/pull/51536)) - More informative back-pressure progress bar ([#51697](https://github.com/ray-project/ray/pull/51697 _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.45.0]_ ### 2.44.1 — Ray-2.44.1 - Date: 2025-03-27 - Version: 2.44.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.44.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.44.1 There is no difference between 2.44.1 and 2.44.0, though we needed a patch version for other out of band reasons. To fill the awkward blankness, here is a haiku about Ray: Under screen-lit skies A ray of bliss in each patch Joy at any scale ### 2.44.0 — Ray-2.44.0 - Date: 2025-03-21 - Version: 2.44.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.44.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.44.0 - **added** — Ray Compiled Graph (beta) with less than 50us system overhead for repeatedly executed task graphs and native GPU-GPU communication via NCCL - **added** — Iceberg write support through pyiceberg in Ray Data - **added** — LoRA support and structured outputs enhancements to Ray Data LLM - **added** — State export API in Ray Train - **added** — Faster bulk imperative Serve Application deploys - **added** — Gen-config for Ray Serve LLM - **added** — Dataset/operator state, progress, and total metrics in Ray Data - **changed** — Make chunk combination threshold configurable in Ray Data - **changed** — Store average memory use per task in OpRuntimeMetrics - **changed** — Avoid unnecessary conversion to Numpy when creating Arrow/Pandas blocks - **changed** — Add append-mode API for preprocessors with vectorizers and hashers now outputting a single column instead of 1 column per feature - **changed** — Fold v2.XGBoostTrainer API into the public trainer class as an alternate constructor - **changed** — Create a default ScalingConfig if one is not provided to the trainer - **changed** — Improve TrainingFailedError message in Ray Train - **changed** — Clean up shutdown behavior of serve - **changed** — Add additional_log_standard_attrs to serve logging config - **fixed** — Fix Map Operators to avoid unconditionally overriding generator's back-pressure configuration - **fixed** — Fix filter expr equating negative numbers in Ray Data - **fixed** — Fix StandardScaler to handle NaN stats in Ray Data - **deprecated** — Ray Workflows library has been deprecated and will be removed in a future version of Ray #### Release Highlights - This release features Ray Compiled Graph (beta). Ray Compiled Graph gives you a classic Ray Core-like API, but with (1) less than 50us system overhead for workloads that repeatedly execute the same task graph; and (2) native support for GPU-GPU communication via NCCL. Ray Compiled Graph APIs simplify high-performance multi-GPU workloads such as LLM inference and training. The beta release refines the API, enhances stability, and adds or improves features like visualization, profiling and experimental GPU compute/computation overlap. For more information, refer to Ray documentation: https://docs.ray.io/en/latest/ray-core/compiled-graph/ray-compiled-graph.html - The experimental Ray Workflows library has been deprecated and will be removed in a future version of Ray. Ray Workflows has been marked experimental since its inception and hasn’t been maintained due to the Ray team focusing on other priorities. If you are using Ray Workflows, we recommend pinning your Ray version to 2.44. #### Ray Libraries ##### Ray Data 🎉 New Features: - Add Iceberg write support through pyiceberg[ (](https://github.com/ray-project/ray/commit/5e26c7fc3866921ce97db876136e04271dabf8b4)[#50590](https://github.com/ray-project/ray/pull/50590)[)](https://github.com/ray-project/ray/commit/5e26c7fc3866921ce97db876136e04271dabf8b4) - [LLM] Various feature enhancements to Ray Data LLM, including LoRA support #50804 and structured outputs #50901 💫 Enhancements: - Add dataset/operator state, progress, total metrics ([#50770](https://github.com/ray-project/ray/pull/50770)) - Make chunk combination threshold configurable ([#51200](https://github.com/ray-project/ray/pull/51200)) - Store average memory use per task in OpRuntimeMetrics ([#51126](https://github.com/ray-project/ray/pull/51126)) - Avoid unnecessary conversion to Numpy when creating Arrow/Pandas blocks ([#51238](https://github.com/ray-project/ray/pull/51238)) - Append-mode API for preprocessors -- #50848, #50847, #50642, #50856, #50584. Note that vectorizers and hashers now output a single column instead 1 column per feature. In the near future, we will be graduating preprocessors to *beta*. 🔨 Fixes: - Fixing Map Operators to avoid unconditionally overriding generator's back-pressure configuration ([#50900](https://github.com/ray-project/ray/pull/50900)) - Fix filter expr equating negative numbers ([#50932](https://github.com/ray-project/ray/pull/50932)) - Fix error message for `override_num_blocks` when reading from a HuggingFace Dataset ([#50998](https://github.com/ray-project/ray/pull/50998)) - Make num_blocks in repartition optional ([#50997](https://github.com/ray-project/ray/pull/50997)) - Always pin the seed when doing file-based random shuffle ([#50924](https://github.com/ray-project/ray/pull/50924)) - Fix `StandardScaler` to handle `NaN` stats ([#51281](https://github.com/ray-project/ray/pull/51281)) ##### Ray Train 🎉 New Features: - Implement state export API (#50622, #51085, #51177) 💫 Enhancements: - Folded v2.XGBoostTrainer API into the public trainer class as an alternate constructor (#50045) - Created a default ScalingConfig if one is not provided to the trainer (#51093) - Improved TrainingFailedError message (#51199) - Utilize FailurePolicy factory (#51067) 🔨 Fixes: - Fixed trainer import deserialization when captured within a Ray task (#50862) - Fixed serialize import test for Python 3.12 (#50963) - Fixed RunConfig deprecation message in Tune being emitted in trainer.fit usage (#51198) 📖 Documentation: - [Train V2] Updated API references (#51222) - [Train V2] Updated persistent storage guide (#51202) - [Train V2] Updated user guides for metrics, checkpoints, results, and experiment tracking (#51204) - [Train V2] Added updated Train + Tune user guide (#51048) - [Train V2] Added updated fault tolerance user guide (#51083) - Improved HF Transformers example (#50896) - Improved Train DeepSpeed ex _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.44.0]_ ### 2.43.0 — Ray-2.43.0 - Date: 2025-02-27 - Version: 2.43.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.43.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.43.0 - **added** — New module ray.data.llm for batch inference with large language models, featuring vLLMEngineProcessorConfig and HttpRequestProcessorConfig abstractions (alpha) - **added** — New module ray.serve.llm for serving large language models - **added** — Ray Train V2 available for testing with RAY_TRAIN_V2_ENABLED=1 environment variable - **added** — Integration with uv run for specifying Python dependencies for Ray cluster drivers and workers - **added** — Support class constructor args for filter() - **changed** — Implement accurate memory accounting for UnionOperator - **changed** — Implement accurate memory accounting for all-to-all operations - **changed** — Persist ParquetDatasource metadata - **changed** — Pass executor as an argument to ExecutionCallback - **changed** — Replace AggregateFn with AggregateFnV2 - **changed** — Add state tracking to train v2 to make run status, run attempts, and training worker metadata observable - **fixed** — Avoid memory leak with pyarrow.infer_type on datetime arrays - **fixed** — Fix parquet partition cols to support tensors types - **fixed** — Fix aggregation protocol to be appropriately associative #### Highlights - This release features new modules in Ray Serve and Ray Data for integration with large language models, marking the first step of addressing [#50639](https://github.com/ray-project/ray/issues/50639). Existing Ray Data and Ray Serve have limited support for LLM deployments, where users have to manually configure and manage the underlying LLM engine. In this release, we offer APIs for both batch inference and serving of LLMs within Ray in `ray.data.llm` and `ray.serve.llm`. See the below notes for more details. These APIs are marked as **alpha** -- meaning they may change in future releases without a deprecation period. - Ray Train V2 is available to try starting in Ray 2.43! Run your next Ray Train job with the `RAY_TRAIN_V2_ENABLED=1` environment variable. See [the migration guide](https://github.com/ray-project/ray/issues/49454) for more information. - A new integration with `uv run` that allows easily specifying Python dependencies for both driver and workers in a consistent way and enables quick iterations for development of Ray applications ([#50160](https://github.com/ray-project/ray/pull/50160), [50462](https://github.com/ray-project/ray/pull/50462)), check out our [blog post](https://www.anyscale.com/blog/uv-ray-pain-free-python-dependencies-in-clusters) #### Ray Libraries ##### Ray Data 🎉 New Features: - *Ray Data LLM*: We are introducing a new module in Ray Data for batch inference with LLMs (currently marked as **alpha**). It offers a new `Processor` abstraction that interoperates with existing Ray Data pipelines. This abstraction can be configured two ways: - Using the `vLLMEngineProcessorConfig`, which configures vLLM to load model replicas for high throughput model inference - Using the `HttpRequestProcessorConfig`, which sends HTTP requests to an OpenAI-compatible endpoint for inference. - Documentation for these features can be [found here.](https://docs.ray.io/en/master/data/working-with-llms.html) - Implement accurate memory accounting for `UnionOperator` ([#50436](https://github.com/ray-project/ray/pull/50436)) - Implement accurate memory accounting for all-to-all operations ([#50290](https://github.com/ray-project/ray/pull/50290)) 💫 Enhancements: - Support class constructor args for filter() ([#50245](https://github.com/ray-project/ray/pull/50245)) - Persist ParquetDatasource metadata. ([#50332](https://github.com/ray-project/ray/pull/50332)) - Rebasing `ShufflingBatcher` onto `try_combine_chunked_columns` ([#50296](https://github.com/ray-project/ray/pull/50296)) - Improve warning message if required dependency isn't installed ([#50464](https://github.com/ray-project/ray/pull/50464)) - Move data-related test logic out of core tests directory ([#50482](https://github.com/ray-project/ray/pull/50482)) - Pass executor as an argument to ExecutionCallback ([#50165](https://github.com/ray-project/ray/pull/50165)) - Add operator id info to task+actor ([#50323](https://github.com/ray-project/ray/pull/50323)) - Abstracting common methods, removing duplication in `ArrowBlockAccessor`, `PandasBlockAccessor` ([#50498](https://github.com/ray-project/ray/pull/50498)) - Warn if map UDF is too large ([#50611](https://github.com/ray-project/ray/pull/50611)) - Replace `AggregateFn` with `AggregateFnV2`, cleaning up Aggregation infrastructure ([#50585](https://github.com/ray-project/ray/pull/50585)) - Simplify Operator.__repr__ ([#50620](https://github.com/ray-project/ray/pull/50620)) - Adding in `TaskDurationStats` and `on_execution_step` callback ([#50766](https://github.com/ray-project/ray/pull/50766)) - Print Resource Manager stats in release tests ([#50801](https://github.com/ray-project/ray/pull/50801)) 🔨 Fixes: - Fix invalid escape sequences in `grouped_data.py` docstrings ([#50392](https://github.com/ray-project/ray/pull/50392)) - Deflake `test_map_batches_async_generator` ([#50459](https://github.com/ray-proj _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.43.0]_ ### 2.42.1 — Ray-2.42.1 - Date: 2025-02-11 - Version: 2.42.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.42.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.42.1 - **fixed** — Incorrect assertion in Ray Data ##### Ray Data 🔨 Fixes: - Fixes incorrect assertion (#50210) ### 2.42.0 — Ray-2.42.0 - Date: 2025-02-05 - Version: 2.42.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.42.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.42.0 - **added** — Add read_audio and read_video for Ray Data - **changed** — Optimize multi-column groupbys in Ray Data - **changed** — Include Ray user-agent in BigQuery client construction - **fixed** — Fix bug that made read tasks non-deterministic in Ray Data - **deprecated** — Deprecate num_rows_per_file in favor of min_rows_per_file in Ray Data - **added** — Add Train v2 user-facing callback interface in Ray Train - **added** — Add TuneReportCallback for propagating intermediate Train results to Tune - **changed** — Cache metrics in replica and report on an interval in Ray Serve - **changed** — Cache expensive calls to inspect.signature in Ray Serve - **changed** — Remove extra pickle serialization for gRPCRequest in Ray Serve - **changed** — Implement shared LongPollClient for Routers in Ray Serve - **changed** — Mark DeploymentHandle API as stable in Ray Serve - **fixed** — Fix batched requests hanging after request cancellation bug in Ray Serve - **added** — Add metrics to replay buffers in RLlib - **changed** — Enhance node-failure tolerance for new API stack in RLlib - **changed** — Improve MetricsLogger cleanup throughput logic in RLlib - **changed** — Split AddStates connectors into AddTimeDimToBatchAndZeroPad and AddStatesFromEpisodesToBatch in RLlib - **fixed** — Re-introduce mixin-replay-buffer pass for old API stack IMPALA/APPO even if replay-ratio=0 - **fixed** — Fix MetricsLogger race conditions in RLlib - **fixed** — Fix APPO/IMPALA bug for greater than 1 Learner actor in RLlib #### Ray Libraries ##### Ray Data 🎉 New Features: - Added read_audio and read_video ([#50016](https://github.com/ray-project/ray/pull/50016)) 💫 Enhancements: - Optimized multi-column groupbys ([#45667](https://github.com/ray-project/ray/pull/45667)) - Included Ray user-agent in BigQuery client construction ([#49922](https://github.com/ray-project/ray/pull/49922)) 🔨 Fixes: - Fixed bug that made read tasks non-deterministic ([#49897](https://github.com/ray-project/ray/pull/49897)) 🗑️ Deprecations: - Deprecated num_rows_per_file in favor of min_rows_per_file ([#49978](https://github.com/ray-project/ray/pull/49978)) ##### Ray Train 💫 Enhancements: - Add Train v2 user-facing callback interface (#49819) - Add TuneReportCallback for propagating intermediate Train results to Tune (#49927) ##### Ray Tune 📖 Documentation: - Fix BayesOptSearch docs (#49848) ##### Ray Serve 💫 Enhancements: - Cache metrics in replica and report on an interval ([#49971](https://github.com/ray-project/ray/pull/49971)) - Cache expensive calls to inspect.signature ([#49975](https://github.com/ray-project/ray/pull/49975)) - Remove extra pickle serialization for gRPCRequest ([#49943](https://github.com/ray-project/ray/pull/49943)) - Shared LongPollClient for Routers ([#48807](https://github.com/ray-project/ray/pull/48807)) - DeploymentHandle API is now stable ([#49840](https://github.com/ray-project/ray/pull/49840)) 🔨 Fixes: - Fix batched requests hanging after request cancellation bug ([#50054](https://github.com/ray-project/ray/pull/50054)) ##### RLlib 💫 Enhancements: - Add metrics to replay buffers. ([#49822](https://github.com/ray-project/ray/pull/49822)) - Enhance node-failure tolerance (new API stack). ([#50007](https://github.com/ray-project/ray/pull/50007)) - MetricsLogger cleanup throughput logic. ([#49981](https://github.com/ray-project/ray/pull/49981)) - Split AddStates... connectors into 2 connector pieces (`AddTimeDimToBatchAndZeroPad` and `AddStatesFromEpisodesToBatch`) ([#49835](https://github.com/ray-project/ray/pull/49835)) 🔨 Fixes: - Old API stack IMPALA/APPO: Re-introduce mixin-replay-buffer pass, even if `replay-ratio=0` (fixes a memory leak). ([#49964](https://github.com/ray-project/ray/pull/49964)) - Fix MetricsLogger race conditions. ([#49888](https://github.com/ray-project/ray/pull/49888)) - APPO/IMPALA: Bug fix for > 1 Learner actor. ([#49849](https://github.com/ray-project/ray/pull/49849)) 📖 Documentation: - New MetricsLogger API rst page. ([#49538](https://github.com/ray-project/ray/pull/49538)) - Move "new API stack" info box right below page titles for better visibility. ([#49921](https://github.com/ray-project/ray/pull/49921)) - Add example script for how to log custom metrics in `training_step()`. ([#49976](https://github.com/ray-project/ray/pull/49976)) - Enhance/redo autoregressive action distribution example. ([#49967](https://github.com/ray-project/ray/pull/49967)) - Make the "tiny CNN" example RLModule run with APPO (by implementing `TargetNetAPI`) ([#49825](https://github.com/ray-project/ray/pull/49825)) #### Ray Core and Ray Clusters ##### Ray Core 💫 Enhancements: - Only get single node info rather then all when needed ([#49727](https://github.com/ray-project/ray/pull/49727)) - Introduce with_tensor_transport API ([#49753](https://github.com/ray-project/ray/pull/49753)) 🔨 Fixes: - Fix tqdm manager thread safe [#50040](https://github.com/ray-project/ray/pull/50040) ##### Ray Clusters 🔨 Fixes: - Fix token expiration for ray autoscaler ([#48481](https://github.com/ray-project/ray/pull/48481)) #### Thanks Thank you to everyone who contributed to this release! 🥳 @wingkitlee0, @saihaj, @win5923, @justinvyu, @kevin85421, @edoakes, @cristianjd, @rynewang, _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.42.0]_ ### 2.41.0 — Ray-2.41.0 - Date: 2025-01-23 - Version: 2.41.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.41.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.41.0 - **added** — Expression support for filters in Ray Data - **added** — Support partition_cols in write_parquet for Ray Data - **added** — Multi-directional sort over Ray Data datasets - **added** — Output JSON structured log files for system and application logs in Ray Train - **added** — Support for AMD ROCR_VISIBLE_DEVICES in Ray Train - **added** — Allow users to configure optuna storage in optuna_search - **added** — Support for external Envs to new API stack in RLlib with custom tcp-capable EnvRunner - **added** — Sequence sampling to EpisodeReplayBuffer in offline RL - **added** — ExecutionCallback interface in Ray Data - **added** — Seed for read files in Ray Data - **added** — Support for individual lambda-style callbacks in RLlib Callbacks API - **changed** — Use dask 2022.10.2 in Ray Data - **changed** — Upgrade Arrow version from 17 to 18 in Ray Data - **changed** — Update hudi version to 0.2.0 in Ray Data - **changed** — Make select_columns and rename_columns use Project operator in Ray Data - **changed** — Improved request_id generation to reduce proxy CPU overhead in Ray Serve - **changed** — Tune GC threshold by default in proxy in Ray Serve - **changed** — Use pickle.dumps for faster serialization from proxy to replica in Ray Serve - **fixed** — Fix groupby hang when value contains np.nan in Ray Data - **fixed** — Fix bug where file_extensions doesn't work with compound extensions in Ray Data - **deprecated** — Default behavior for sync methods will change in a future release to run in a threadpool by default in Ray Serve #### Highlights - Major update of RLlib docs and example scripts for the new API stack. #### Ray Libraries ##### Ray Data 🎉 New Features: - Expression support for filters (#49016) - Support `partition_cols` in `write_parquet` (#49411) - Feature: implement multi-directional sort over Ray Data datasets (#49281) 💫 Enhancements: - Use dask 2022.10.2 (#48898) - Clarify schema validation error (#48882) - Raise `ValueError` when the data sort key is `None` (#48969) - Provide more messages when webdataset format is error (#48643) - Upgrade Arrow version from 17 to 18 (#48448) - Update `hudi` version to 0.2.0 (#48875) - `webdataset`: expand JSON objects into individual samples (#48673) - Support passing kwargs to map tasks. (#49208) - Add `ExecutionCallback` interface (#49205) - Add seed for read files (#49129) - Make `select_columns` and `rename_columns` use Project operator (#49393) 🔨 Fixes: - Fix partial function name parsing in `map_groups` (#48907) - Always launch one task for `read_sql` (#48923) - Reimplement of fix memory pandas (#48970) - `webdataset`: flatten return args (#48674) - Handle `numpy > 2.0.0` behaviour in `_create_possibly_ragged_ndarray` (#48064) - Fix `DataContext` sealing for multiple datasets. (#49096) - Fix `to_tf` for `List` types (#49139) - Fix type mismatch error while mapping nullable column (#49405) - Datasink: support passing write results to `on_write_completes` (#49251) - Fix `groupby` hang when value contains `np.nan` (#49420) - Fix bug where `file_extensions` doesn't work with compound extensions (#49244) - Fix map operator fusion when concurrency is set (#49573) ##### Ray Train 🎉 New Features: - Output JSON structured log files for system and application logs (#49414) - Add support for AMD ROCR_VISIBLE_DEVICES (#49346) 💫 Enhancements: - Implement Train Tune API Revamp REP (#49376, #49467, #49317, #49522) 🏗 Architecture refactoring: - LightGBM: Rewrite `get_network_params` implementation (#49019) ##### Ray Tune 🎉 New Features: - Update `optuna_search` to allow users to configure optuna storage (#48547) 🏗 Architecture refactoring: - Make changes to support Train Tune API Revamp REP (#49308, #49317, #49519) ##### Ray Serve 💫 Enhancements: - Improved request_id generation to reduce proxy CPU overhead (#49537) - Tune GC threshold by default in proxy (#49720) - Use `pickle.dumps` for faster serialization from `proxy` to `replica` (#49539) 🔨 Fixes: - Handle nested ‘=’ in serve run arguments (#49719) - Fix bug when `ray.init()` is called multiple times with different `runtime_envs` (#49074) 🗑️ Deprecations: - Adds a warning that the default behavior for sync methods will change in a future release. They will be run in a threadpool by default. You can opt into this behavior early by setting `RAY_SERVE_RUN_SYNC_IN_THREADPOOL=1`. (#48897) ##### RLlib 🎉 New Features: - Add support for external Envs to new API stack: New example script and custom tcp-capable EnvRunner. (#49033) 💫 Enhancements: - Offline RL: - Add sequence sampling to `EpisodeReplayBuffer`. (#48116) - Allow incomplete `SampleBatch` data and fully compressed observations. (#48699) - Add option to customize `OfflineData`. (#49015) - Enable offline training without specifying an environment. (#49041) - Various fixes: #48309, #49194, #49195 - APPO/IMPALA acceleration (new API stack): - Add support for `AggregatorActors` per Learner. (#49284) - Auto-sleep time AND thread-safety for MetricsLogger. (#48868) - Activate APPO cont. actions release- and CI tests (HalfCheetah-v1 and Pendulum-v1 new in `tuned_examples`). (#49068) - Add "burn-in" period setting to the training of stateful RLModules. (#49680) - Callbacks API: Add support for individual lambda-style callbacks. (#49511) - Other enhancements: #49687, #49714, #49693, #49497, #49800, #49098 📖 Documentation: - New _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.41.0]_ ### 2.40.0 — Ray-2.40.0 - Date: 2024-12-04 - Version: 2.40.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.40.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.40.0 - **added** — Add read_hudi function to Ray Data - **changed** — Improve performance of DelegatingBlockBuilder - **changed** — Improve memory accounting of pandas blocks - **fixed** — Fix bug where you can't specify a schema with write_parquet - **fixed** — Fix bug where to_pandas errors if your dataset contains Arrow and pandas blocks - **fixed** — Fix bug where map_groups doesn't work with pandas data - **fixed** — Fix bug where write_parquet errors if your data contains nullable fields - **fixed** — Fix bug where "Iteration Blocked Time" charts looks incorrect - **fixed** — Fix bug where unique fails with null values - **fixed** — Fix bug where "Rows Outputted" is 0 in the Data dashboard - **fixed** — Fix bug where methods like drop_columns cause spilling - **fixed** — Fix bug where async map tasks hang - **deprecated** — Deprecate read_parquet_bulk - **deprecated** — Deprecate iter_tf_batches - **deprecated** — Deprecate meta_provider parameter of read functions - **deprecated** — Deprecate to_torch - **fixed** — Fix StartTracebackWithWorkerRank serialization in Ray Train - **added** — Add initial version of local_testing_mode to Ray Serve - **changed** — Handle multiple changed objects per LongPollHost.listen_for_change RPC in Ray Serve - **added** — Add uv runtime env support to Ray Core #### Ray Libraries ##### Ray Data 🎉 New Features: - Added read_hudi (https://github.com/ray-project/ray/pull/46273) 💫 Enhancements: - Improved performance of DelegatingBlockBuilder (https://github.com/ray-project/ray/pull/48509) - Improved memory accounting of pandas blocks (https://github.com/ray-project/ray/pull/46939) 🔨 Fixes: - Fixed bug where you can’t specify a schema with write_parquet (https://github.com/ray-project/ray/issues/48630) - Fixed bug where to_pandas errors if your dataset contains Arrow and pandas blocks (https://github.com/ray-project/ray/pull/48583) - Fixed bug where map_groups doesn’t work with pandas data (https://github.com/ray-project/ray/pull/48287) - Fixed bug where write_parquet errors if your data contains nullable fields (https://github.com/ray-project/ray/pull/48478) - Fixed bug where “Iteration Blocked Time” charts looks incorrect (https://github.com/ray-project/ray/pull/48618) - Fixed bug where unique fails with null values (https://github.com/ray-project/ray/pull/48750) - Fixed bug where “Rows Outputted” is 0 in the Data dashboard (https://github.com/ray-project/ray/pull/48745) - Fixed bug where methods like drop_columns cause spilling (https://github.com/ray-project/ray/pull/48140) - Fixed bug where async map tasks hang (https://github.com/ray-project/ray/pull/48861) 🗑️ Deprecations: - Deprecated read_parquet_bulk https://github.com/ray-project/ray/pull/48691 - Deprecated iter_tf_batches https://github.com/ray-project/ray/pull/48693 - Deprecated meta_provider parameter of read functions (https://github.com/ray-project/ray/pull/48690) - Deprecated to_torch (https://github.com/ray-project/ray/pull/48692) ##### Ray Train 🔨 Fixes: - Fix StartTracebackWithWorkerRank serialization (#48548) 📖 Documentation: - Add example for fine-tuning Llama3.1 with AWS Trainium (#48768) ##### Ray Tune 🔨 Fixes: - Remove the `clear_checkpoint` function during Trial restoration error handling. (#48532) ##### Ray Serve 🎉 New Features: - Initial version of local_testing_mode ([#48477](https://github.com/ray-project/ray/pull/48477)) 💫 Enhancements: - Handle multiple changed objects per LongPollHost.listen_for_change RPC ([#48803](https://github.com/ray-project/ray/pull/48803/files)) - Add more nuanced checks for http proxy status errors ([#47896](https://github.com/ray-project/ray/pull/47896)) - Improve replica access log messages to include HTTP status info and better resemble standard log format ([#48819](https://github.com/ray-project/ray/pull/48819)) - Propagate replica constructor error to deployment status message and print num retries left ([#48531](https://github.com/ray-project/ray/pull/47896)) 🔨 Fixes: - Pending requests that are cancelled before they were assigned to a replica now also return a serve.RequestCancelledError ([#48496](https://github.com/ray-project/ray/pull/48496)) ##### RLlib 💫 Enhancements: - Release test enhancements. ([#45803](https://github.com/ray-project/ray/pull/45803), [#48681](https://github.com/ray-project/ray/pull/48681)) - Make opencv-python-headless default over opencv-python ([#48776](https://github.com/ray-project/ray/pull/48776)[)](https://github.com/ray-project/ray/commit/aaac19c8307038021dd96ffc4c2e616fbbf14896) - Reverse learner queue behavior of IMPALA/APPO (consume oldest batches first, instead of newest, BUT drop oldest batches if queue full). ([#48702](https://github.com/ray-project/ray/pull/48702)) 🔨 Fixes: - Fix torch scheduler stepping and reporting. ([#48125](https://github.com/ray-project/ray/pull/48125)[)](https://github.com/ray-project/ray/commit/ec9775d86fbf7eb93358d95268e9f62e53f790bd) - Fix accumulation of results over n training_step calls within same iteration (new API stack). ([#48136](https://github.com/ray-project/ray/pull/48136)) - Various other fixes: [#48563](https://github.com/ray-project/ray/pull/48563), [#48314](https://github.com/ray-project/ray/pull/48314), [#48698](h _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.40.0]_ ### 2.39.0 — Ray-2.39.0 - Date: 2024-11-13 - Version: 2.39.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.39.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.39.0 - **fixed** — Fixed InvalidObjectError edge case with Dataset.split() - **fixed** — Made Concatenator preserve order of concatenated columns - **changed** — Marked num_rows_per_file parameter of write APIs as experimental - **fixed** — transform_batch no longer fails on missing columns - **changed** — Dataset.count() now uses a Count logical operator - **removed** — Removed long-deprecated set_progress_bars - **fixed** — Safely check if the storage filesystem is pyarrow.fs.S3FileSystem - **added** — Cancelled requests now return a serve.RequestCancelledError - **added** — Exposed application source in app details model - **fixed** — Basic HTTP deployments will now return "Internal Server Error" instead of a traceback to match FastAPI behavior - **fixed** — Fixed an issue where high values of max_ongoing_requests couldn't be reached due to an interaction with core's max_concurrency - **fixed** — Fixed an edge case where pending requests were not canceled properly - **removed** — Removed deprecated API to set route_prefix per-deployment - **changed** — Upgrade to gymnasium==1.0.0 with support for new API for vector env resets - **added** — Add off-policy'ness metric to new API stack - **added** — Validate episodes before adding them to the buffer - **added** — Support all reduce collective in aDAG - **added** — Add visualization of compiled graphs - **changed** — The distributed debugger can now be used without having to set RAY_DEBUG=1 - **added** — Support asyncio.gather on multiple CompiledDAGFutures #### Ray Libraries ##### Ray Data 🔨 Fixes: - Fixed InvalidObjectError edge case with Dataset.split() (https://github.com/ray-project/ray/pull/48130) - Made Concatenator preserve order of concatenated columns (https://github.com/ray-project/ray/pull/47997) 📖 Documentation: - Improved documentation around Parquet column and predicate pushdown (https://github.com/ray-project/ray/pull/48095) - Marked num_rows_per_file parameter of write APIs as experimental (https://github.com/ray-project/ray/pull/48208) - One hot encoder now returns an encoded vector (https://github.com/ray-project/ray/pull/48173) - transform_batch no longer fails on missing columns (https://github.com/ray-project/ray/pull/48137) 🏗 Architecture refactoring: - Dataset.count() now uses a Count logical operator (https://github.com/ray-project/ray/pull/48126) 🗑 Deprecations: - Removed long-deprecated set_progress_bars (https://github.com/ray-project/ray/pull/48203) ##### Ray Train 🔨 Fixes: - Safely check if the storage filesystem is `pyarrow.fs.S3FileSystem` (#48216) ##### Ray Tune 🔨 Fixes: - Safely check if the storage filesystem is `pyarrow.fs.S3FileSystem` (#48216) ##### Ray Serve 💫 Enhancements: - Cancelled requests now return a serve.RequestCancelledError (https://github.com/ray-project/ray/pull/48444) - Exposed application source in app details model (https://github.com/ray-project/ray/pull/45522) 🔨 Fixes: - Basic HTTP deployments will now return “Internal Server Error” instead of a traceback to match FastAPI behavior (https://github.com/ray-project/ray/pull/48491) - Fixed an issue where high values of max_ongoing_requests couldn’t be reached due to an interaction with core’s max_concurrency (https://github.com/ray-project/ray/pull/48274) - Fixed an edge case where pending requests were not canceled properly (https://github.com/ray-project/ray/pull/47873) - Removed deprecated API to set route_prefix per-deployment (https://github.com/ray-project/ray/pull/48223) 📖 Documentation: - Added ProxyStatus model to reference docs (https://github.com/ray-project/ray/pull/48299) - Added ApplicationStatus model to reference docs (https://github.com/ray-project/ray/pull/48220) ##### RLlib 💫 Enhancements: - Upgrade to gymnasium==1.0.0 (support new API for vector env resets). ([#48443](https://github.com/ray-project/ray/pull/48443), [#45328](https://github.com/ray-project/ray/pull/45328)) - Add off-policy'ness metric to new API stack. ([#48227](https://github.com/ray-project/ray/pull/48227)) - Validate episodes before adding them to the buffer. ([#48083](https://github.com/ray-project/ray/pull/48083)) 📖 Documentation: - New example script for custom metrics on `EnvRunners` (using `MetricsLogger` API on the new stack). ([#47969](https://github.com/ray-project/ray/pull/47969)) - Do-over: New RLlib index page. ([#48285](https://github.com/ray-project/ray/pull/48285), [#48442](https://github.com/ray-project/ray/pull/48442)) - Do-over: Example script for AutoregressiveActionsRLM. ([#47972](https://github.com/ray-project/ray/pull/47972)) 🏗 Architecture refactoring: - New API stack on by default for PPO. ([#48284](https://github.com/ray-project/ray/pull/48284)) - Change config.fault_tolerance default behavior (from `recreate_failed_env_runners=False` to `True`). ([#48286](https://github.com/ray-project/ray/pull/48286)) 🔨 Fixes: - Various bug and CI fixes: [#47993](https://github.com/ray-project/ray/pull/47993), [#48450](https://github.com/ray-project/ray/pull/48450), [#48213](https://github.com/ray-project/ray/pull/48213) - Cleanup `evaluation` folder ([#48493](https://github.com/ray-project/ray/pull/48493)) ##### Ray Core 🎉 New Features: - [CompiledGraphs] Support all reduce collective in aDAG ([#47621](https://github.com/ray-project/ray/pull/47621)) - [CompiledGraphs] Add visualization of compiled graphs ([#47958](https://github.com/ray-project/ray/pull/47958)) 💫 Enhancement _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.39.0]_ ### 2.38.0 — Ray-2.38.0 - Date: 2024-10-23 - Version: 2.38.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.38.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.38.0 - **added** — Add Dataset.rename_columns - **added** — Add basic structured logging to Ray Data - **added** — Add partitioning parameter to read_parquet - **added** — Enable cloud checkpointing in RLlib - **added** — CompiledGraphs: support multi readers in multi node when DAG is created from an actor - **changed** — Add SERVICE_UNAVAILABLE to list of retried transient errors - **changed** — Re-phrase the streaming executor current usage string - **changed** — Remove ray.kill in ActorPoolMapOperator - **changed** — Simplify and consolidate progress bar outputs - **changed** — Allow specifying both num_cpus and num_gpus for map APIs - **changed** — Allow specifying insertion index when registering custom plan optimization Rules - **changed** — PPO on new API stack now shuffles batches properly before each epoch - **changed** — Switch on new API stack by default for SAC and DQN - **changed** — Remove TensorFlow support on new API stack for PPO/IMPALA/APPO - **changed** — Discontinue support for hybrid API stack - **changed** — Store each GCS table in its own Redis Hash - **changed** — Decouple create worker vs pop worker request - **fixed** — Fix bug where Ray Data incorrectly emits progress bar warning - **fixed** — Yield remaining results from async map_batches - **fixed** — Fix event loop mismatch with async map #### Ray Libraries ##### Ray Data 🎉 New Features: - Add `Dataset.rename_columns` (#47906) - Basic structured logging (#47210) 💫 Enhancements: - Add `partitioning` parameter to `read_parquet` (#47553) - Add `SERVICE_UNAVAILABLE` to list of retried transient errors (#47673) - Re-phrase the streaming executor current usage string (#47515) - Remove ray.kill in ActorPoolMapOperator (#47752) - Simplify and consolidate progress bar outputs (#47692) - Refactor `OpRuntimeMetrics` to support properties (#47800) - Refactor `plan_write_op` and `Datasink`s (#47942) - Link `PhysicalOperator` to its `LogicalOperator` (#47986) - Allow specifying both `num_cpus` and `num_gpus` for map APIs (#47995) - Allow specifying insertion index when registering custom plan optimization `Rule`s (#48039) - Adding in better framework for substituting logging handlers (#48056) 🔨 Fixes: - Fix bug where Ray Data incorrectly emits progress bar warning (#47680) - Yield remaining results from async `map_batches` (#47696) - Fix event loop mismatch with async map (#47907) - Make sure `num_gpus` provide to Ray Data is appropriately passed to `ray.remote` call (#47768) - Fix unequal partitions when grouping by multiple keys (#47924) - Fix reading multiple parquet files with ragged ndarrays (#47961) - Removing unneeded test case (#48031) - Adding in better json checking in test logging (#48036) - Fix bug with inserting custom optimization rule at index 0 (#48051) - Fix logging output from `write_xxx` APIs (#48096) 📖 Documentation: - Add docs section for Ray Data progress bars (#47804) - Add reference to parquet predicate pushdown (#47881) - Add tip about how to understand map_batches format (#47394) ##### Ray Train 🏗 Architecture refactoring: - Remove deprecated mosaic and sklearn trainer code (#47901) ##### Ray Tune 🔨 Fixes: - Fix WandbLoggerCallback to reuse actors upon restore (#47985) ##### Ray Serve 🔨 Fixes: - Stop scheduling task early when requests have been canceled (#47847) ##### RLlib 🎉 New Features: - Enable cloud checkpointing. (#47682) 💫 Enhancements: - PPO on new API stack now shuffles batches properly before each epoch. (#47458) - Other enhancements: #47705, #47501, #47731, #47451, #47830, #47970, #47157 🔨 Fixes: - Fix spot node preemption problem (RLlib now run stably with EnvRunner workers on spot nodes) (#47940) - Fix action masking example. (#47817) - Various other fixes: #47973, #46721, #47914, #47880, #47304, #47686 🏗 Architecture refactoring: - Switch on new API stack by default for SAC and DQN. (#47217) - Remove Tf support on new API stack for PPO/IMPALA/APPO (only DreamerV3 on new API stack remains with tf now). (#47892) - Discontinue support for "hybrid" API stack (using RLModule + Learner, but still on RolloutWorker and Policy) (#46085) - RLModule (new API stack) refinements: #47884, #47885, #47889, #47908, #47915, #47965, #47775 📖 Documentation: - Add new API stack migration guide. (#47779) - New API stack example script: BC pre training, then PPO finetuning using same RLModule class. (#47838) - New API stack: Autoregressive actions example. (#47829) - Remove old API stack connector docs entirely. (#47778) #### Ray Core and Ray Clusters ##### Ray Core 🎉 New Features: - CompiledGraphs: support multi readers in multi node when DAG is created from an actor (#47601) 💫 Enhancements: - Add a flag to raise exception for out of band serialization of `ObjectRef` (#47544) - Store each GCS table in its own Redis Hash (#46861) - Decouple create worker vs pop worker request. (#47694) - Add metrics for GCS jobs (#47793) 🔨 Fixes: - Fix broken dashboard cluster page when there are dead nodes (#47701) - Fix the `ray_tasks{State="PENDING_ARGS_FETCH"}` metric counting (#47770) - Separ _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.38.0]_ ### 2.37.0 — Ray-2.37.0 - Date: 2024-09-24 - Version: 2.37.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.37.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.37.0 - **changed** — Simplify custom metadata provider API - **changed** — Change counts of metrics to rates of metrics - **fixed** — Throw exception for non-streaming HF datasets with override_num_blocks argument - **changed** — Refactor custom optimizer rules - **fixed** — Remove ineffective retry code in plan_read_op - **fixed** — Fix incorrect pending task size if outputs are empty - **changed** — Update run status and add stack trace to TrainRunInfo - **added** — Allow control of some serve configuration via env vars - **changed** — Faster detection of dead replicas - **fixed** — Fix component id logging field - **added** — Add restart-failed-env option to EnvRunners - **changed** — Store episodes in state form for offline RL - **changed** — Replace GAE in MARWILOfflinePreLearner with GeneralAdvantageEstimation connector in learner pipeline - **added** — Add episode sampling to EpisodeReplayBuffer - **added** — Add SelfSupervisedLossAPI for RLModules that bring their own loss - **added** — Add InferenceOnlyAPI - **added** — Allow custom NCCL group for aDAG - **added** — Support buffered input for aDAG - **added** — Support multi node multi reader for aDAG - **changed** — Make is_gpu, is_actor, root_detached_id fields late bind to workers - **changed** — Reconstruct actor to run lineage reconstruction triggered actor task #### Ray Libraries ##### Ray Data 💫 Enhancements: - Simplify custom metadata provider API (#47575) - Change counts of metrics to rates of metrics (#47236) - Throw exception for non-streaming HF datasets with "override_num_blocks" argument (#47559) - Refactor custom optimizer rules (#47605) 🔨 Fixes: - Remove ineffective retry code in `plan_read_op` (#47456) - Fix incorrect pending task size if outputs are empty (#47604) ##### Ray Train 💫 Enhancements: - Update run status and add stack trace to `TrainRunInfo` (#46875) ##### Ray Serve 💫 Enhancements: - Allow control of some serve configuration via env vars ([#47533](https://github.com/ray-project/ray/pull/47533)) - [serve] Faster detection of dead replicas ([#47237](https://github.com/ray-project/ray/pull/47237)) 🔨 Fixes: - [Serve] fix component id logging field ([#47609](https://github.com/ray-project/ray/pull/47609)) ##### RLlib 💫 Enhancements: - New API stack: - Add restart-failed-env option to EnvRunners. ([#47608](https://github.com/ray-project/ray/pull/47608)[)](https://github.com/ray-project/ray/commit/e75f5e7aa950e30097a0323f4baf14d90b1b6b9b) - Offline RL: Store episodes in state form. ([#47294](https://github.com/ray-project/ray/pull/47294)[)](https://github.com/ray-project/ray/commit/aa7179a6fa24a0d95a1c9b85014bfb322d3447e6) - Offline RL: Replace GAE in MARWILOfflinePreLearner with `GeneralAdvantageEstimation` connector in learner pipeline. ([#47532](https://github.com/ray-project/ray/pull/47532)) - Off-policy algos: Add episode sampling to EpisodeReplayBuffer. ([#47500](https://github.com/ray-project/ray/pull/47500)) - RLModule APIs: Add `SelfSupervisedLossAPI` for RLModules[ that bri](https://github.com/ray-project/ray/commit/f422376cda3ae0dc52fc7686df3b1cb03342be7f)ng their own loss and `InferenceOnlyAPI`. ([#47581](https://github.com/ray-project/ray/pull/47581), [#47572](https://github.com/ray-project/ray/pull/47572)) ##### Ray Core 💫 Enhancements: - [aDAG] Allow custom NCCL group for aDAG (#47141) - [aDAG] support buffered input (#47272) - [aDAG] Support multi node multi reader (#47480) - [Core] Make is_gpu, is_actor, root_detached_id fields late bind to workers. (#47212) - [Core] Reconstruct actor to run lineage reconstruction triggered actor task (#47396) - [Core] Optimize GetAllJobInfo API for performance (#47530) 🔨 Fixes: - [aDAG] Fix ranks ordering for custom NCCL group (#47594) ##### Ray Clusters 📖 Documentation: - [KubeRay] add a guide for deploying vLLM with RayService (#47038) #### Thanks Many thanks to all those who contributed to this release! @ruisearch42, @andrewsykim, @timkpaine, @rkooo567, @WeichenXu123, @GeneDer, @sword865, @simonsays1980, @angelinalg, @sven1977, @jjyao, @woshiyyya, @aslonnie, @zcin, @omatthew98, @rueian, @khluu, @justinvyu, @bveeramani, @nikitavemuri, @chris-ray-zhang, @liuxsh9, @xingyu-long, @peytondmurray, @rynewang ### 2.36.1 — Ray-2.36.1 - Date: 2024-09-23 - Version: 2.36.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.36.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.36.1 - **fixed** — Fix broken dashboard cluster page when there are dead nodes - **fixed** — Fix broken dashboard worker page ##### Ray Core 🔨 Fixes: - Fix broken dashboard cluster page when there are dead nodes (#47701) - Fix broken dashboard worker page (#47714) ### 2.36.0 — Ray-2.36.0 - Date: 2024-09-17 - Version: 2.36.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.36.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.36.0 - **changed** — Remove limit on number of tasks launched per scheduling step in Ray Data - **added** — Allow user-defined Exception to be caught in Ray Data - **fixed** — Display pending actors separately in the progress bar and not count them towards running resources - **fixed** — Fix bug where arrow_parquet_args aren't used - **fixed** — Skip empty JSON files in read_json() - **fixed** — Remove remote call for initializing Datasource in read_datasource() - **added** — Decouple device-related modules and add Huawei NPU support to Ray Train - **fixed** — Update TORCH_NCCL_ASYNC_ERROR_HANDLING env var - **changed** — Mark proxy as unready when its routers are aware of zero replicas in Ray Serve - **added** — Setup default serve logger - **fixed** — Allow get_serve_logs_dir to run outside of Ray's context - **removed** — Remove support for nested DeploymentResponses - **added** — Add CQL algorithm to new API stack in RLlib - **added** — Enable GPU and multi-GPU support for DQN/SAC/CQL in new API stack - **changed** — Enhance offline RL in new API stack - **fixed** — Fix large batch size for synchronous algos after EnvRunner failures - **added** — Add torch.compile config options to old API stack - **added** — Add kwargs to torch.nn.parallel.DistributedDataParallel - **added** — Add PREPARED state for placement groups in GCS for better fault tolerance - **fixed** — Fix ray_unintentional_worker_failures_total to only count unintentional worker failures #### Ray Libraries ##### Ray Data 💫 Enhancements: - Remove limit on number of tasks launched per scheduling step (#47393) - Allow user-defined Exception to be caught. (#47339) 🔨 Fixes: - Display pending actors separately in the progress bar and not count them towards running resources (#46384) - Fix bug where `arrow_parquet_args` aren't used (#47161) - Skip empty JSON files in `read_json()` (#47378) - Remove remote call for initializing `Datasource` in `read_datasource()` (#47467) - Remove dead `from_*_operator` modules (#47457) - Release test fixes - Add `AWS ACCESS_DENIED` as retryable exception for multi-node Data+Train benchmarks (#47232) - Get AWS credentials with boto (#47352) - Use worker node instead of head node for `read_images_comparison_microbenchmark_single_node` release test (#47228) 📖 Documentation: - Add docstring to explain `Dataset.deserialize_lineage` (#47203) - Add a comment explaining the bundling behavior for `map_batches` with default batch_size (#47433) ##### Ray Train 💫 Enhancements: - Decouple device-related modules and add Huawei NPU support to Ray Train (#44086) 🔨 Fixes: - Update TORCH_NCCL_ASYNC_ERROR_HANDLING env var (#47292) 📖 Documentation: - Add missing Train public API reference (#47134) ##### Ray Tune 📖 Documentation: - Add missing Tune public API references (#47138) ##### Ray Serve 💫 Enhancements: - Mark proxy as unready when its routers are aware of zero replicas (#47002) - Setup default serve logger (#47229) 🔨 Fixes: - Allow get_serve_logs_dir to run outside of Ray's context (#47224) - Use serve logger name for logs in serve (#47205) 📖 Documentation: - [HPU] [Serve] [experimental] Add vllm HPU support in vllm example (#45893) 🏗 Architecture refactoring: - Remove support for nested DeploymentResponses (#47209) ##### RLlib 🎉 New Features: - New API stack: Add CQL algorithm. ([#47000](https://github.com/ray-project/ray/pull/47000), [#47402](https://github.com/ray-project/ray/pull/47402)) - New API stack: Enable GPU and multi-GPU support for DQN/SAC/CQL. ([#47179](https://github.com/ray-project/ray/pull/47179)) 💫 Enhancements: - New API stack: Offline RL enhancements: [#47195](https://github.com/ray-project/ray/pull/47195), [#47359](https://github.com/ray-project/ray/pull/47359) - Enhance new API stack stability: [#46324](https://github.com/ray-project/ray/pull/46324), [#47196](https://github.com/ray-project/ray/pull/47196), [#47245](https://github.com/ray-project/ray/pull/47245), [#47279](https://github.com/ray-project/ray/pull/47279) - Fix large batch size for synchronous algos (e.g. PPO) after EnvRunner failures. ([#47356](https://github.com/ray-project/ray/pull/47356)) - Add torch.compile config options to old API stack. ([#47340](https://github.com/ray-project/ray/pull/47340)[)](https://github.com/ray-project/ray/commit/78402bc1fde669cb4015f9393106acb144ac45bf) - Add kwargs to torch.nn.parallel.DistributedDataParallel ([#47276](https://github.com/ray-project/ray/pull/47276)) - Enhanced CI stability: [#47197](https://github.com/ray-project/ray/pull/47197), [#47249](https://github.com/ray-project/ray/pull/47249) 📖 Documentation: - New API stack example scripts: - Float16 training example script. ([#47362](https://github.com/ray-project/ray/pull/47362)) - Mixed precision training example script ([#47116](https://github.com/ray-project/ray/pull/47116)) - ModelV2 -> RLModule wrapper for migrating to new API stack. ([#47425](https://github.com/ray-project/ray/pull/47425)) - Remove "new API stack experimental" hint from docs. ([#47301](https://github.com/ray-project/ray/pull/47301)) 🏗 Architecture refactoring: - Remove 2nd Learner ConnectorV2 pass from PPO ([#47401](https://github.com/ray-project/ray/pull/47401)) - Add separate learning rates for policy a _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.36.0]_ ### 2.35.0 — Ray-2.35.0 - Date: 2024-08-28 - Version: 2.35.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.35.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.35.0 - **changed** — pip install ray[all] will no longer include ray[cpp] and the ray-cpp package; use pip install ray[cpp-all] to install everything including ray-cpp - **changed** — Upgrade supported Arrow version from 16 to 17 - **added** — Add support for reading from Iceberg in Ray Data - **changed** — Various progress bar UX improvements in Ray Data - **changed** — Try get size_bytes from metadata and consolidate metadata methods in Ray Data - **changed** — Improve warning message when read task is large in Ray Data - **changed** — Extend API to enable passing sample weights via ray.dataset.to_tf - **changed** — Add a parameter to allow overriding LanceDB scanner options in Ray Data - **added** — Add failure retry logic for read_lance in Ray Data - **fixed** — Change type of DataContext.retried_io_errors from tuple to list in Ray Data - **fixed** — Change pickling log level from warning to debug in Ray Data - **fixed** — Add validation for shuffle arg in Ray Data - **fixed** — Fix validation bug when size=0 in ActorPoolStrategy in Ray Data - **fixed** — Fix exception in async map in Ray Data - **fixed** — Fix wrong metrics group for Object Store Memory metrics on Ray Data Dashboard - **fixed** — Handle errors in SplitCoordinator when generating a new epoch in Ray Data - **changed** — Updates to support xgboost==2.1.0 in Ray Train - **added** — Add hardware stats in Ray Train - **fixed** — Fix WandB metric overlap after restore from checkpoint in Tune - **changed** — Improve handling of replica death and replica unavailability in deployment handle routers before controller restarts replica in Ray Serve - **deprecated** — Deprecate passing arguments that contain DeploymentResponses in nested objects to downstream deployment handle calls in Ray Serve **Notice**: Starting from this release, `pip install ray[all]` will not include `ray[cpp]`, and will not install the respective `ray-cpp` package. To install everything that includes `ray-cpp`, one can use `pip install ray[cpp-all]` instead. #### Ray Libraries ##### Ray Data 🎉 New Features: - Upgrade supported Arrow version from 16 to 17 (#47034) - Add support for reading from Iceberg (#46889) 💫 Enhancements: - Various Progress Bar UX improvements (#46816, #46801, #46826, #46692, #46699, #46974, #46928, #47029, #46924, #47120, #47095, #47106) - Try get `size_bytes` from metadata and consolidate metadata methods (#46862) - Improve warning message when read task is large (#46942) - Extend API to enable passing sample weights via ray.dataset.to_tf (#45701) - Add a parameter to allow overriding LanceDB scanner options (#46975) - Add failure retry logic for read_lance (#46976) - Clarify warning for reading old Parquet data (#47049) - Move datasource implementations to `_internal` subpackage (#46825) - Handle logs from tensor extensions (#46943) 🔨 Fixes: - Change type of `DataContext.retried_io_errors` from tuple to list (#46884) - Make Parquet tests more robust and expose Parquet logic (#46944) - Change pickling log level from warning to debug (#47032) - Add validation for shuffle arg (#47055) - Fix validation bug when size=0 in ActorPoolStrategy (#47072) - Fix exception in async map (#47110) - Fix wrong metrics group for `Object Store Memory` metrics on Ray Data Dashboard (#47170) - Handle errors in SplitCoordinator when generating a new epoch (#47176) 📖 Documentation: - Auto-gen GroupedData api (#46925) - Fix signature of `Rule.plan` (#47094) ##### Ray Train 💫 Enhancements: - [train] Updates to support xgboost==2.1.0 (#46667) - [train] Add hardware stats (#46719) ##### Ray Tune 🔨 Fixes: - [RLlib; Tune] Fix WandB metric overlap after restore from checkpoint. (#46897) ##### Ray Serve 💫 Enhancements: - Improved handling of replica death and replica unavailability in deployment handle routers before controller restarts replica (#47008) - Eagerly create routers in proxy for better GCS fault tolerance (#47031) - Immediately send ping in router when receiving new replica set (#47053) 🏗 Architecture refactoring: - Deprecate passing arguments that contain `DeploymentResponses` in nested objects to downstream deployment handle calls (#46806) ##### RLlib 🎉 New Features: - Offline RL on the new API stack: - Record offline data (#46818, #47046, #47133, #47155) and support to directly read from episodes. (#46865) - RLUnplugged example. (#46792) - Progress on BC/MARWIL migration: #44970, #47154, #46799 - Progress on CQL migration: #46969, #47105 💫 Enhancements: - Add ObservationPreprocessor (ConnectorV2). (#47077) 🔨 Fixes: - New API stack: Fix IMPALA/APPO + LSTM for single- and multi-GPU. (#47132, #47158) - Various bug fixes: #46898, #47047, #46963, #47021, #46897 - Add more control to Algorithm.add_module/policy methods. (#46932, #46836) 📖 Documentation: - Example scripts for new API stack: - Curiosity (inverse dynamics model-based) RLModule example. (#46841) - Add example script for Env with protobuf observation space. (#47071) - New API stack documentation: - Cleanup old API stack docs (rllib-dev.rst). (#47172) - Episodes (SingleAgentEpisode). (#46985) - Redo rllib-algorithms.rst page. (#46916) 🏗 Architecture refactoring: - Rename MultiAgent...RLModule... into MultiRL...Module for more generality. (#46840) - Add learner_only flag to RLModuleConfig/Spec and simplify creation of RLModule specs from algo-config. (#46900) ##### Ray Core 💫 Enhancements: - Emit total lineage bytes metrics (#46725) - Adding accelerator type H100 (#46823) - More structured logging in c _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.35.0]_ ### 2.34.0 — Release 2.34.0 Notes - Date: 2024-07-31 - Version: 2.34.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.34.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.34.0 - **added** — Add better support for UDF returns from list of datetime objects - **fixed** — Remove read task warning if size bytes not set in metadata - **fixed** — Sort workers by node ID rather than by node IP - **removed** — Remove dead RayDatasetSpec - **added** — Initial design for Ray-Data based offline RL Algos on new API stack - **added** — Add user-defined schemas for data loading - **added** — Make data pipeline better configurable and tuneable for users - **changed** — Move DQN into the TargetNetworkAPI - **deprecated** — Deprecate RLModuleWithTargetNetworksInterface - **fixed** — Rename all np.product usage to np.prod for numpy compatibility - **changed** — Cleanup, rename, clarify: Algorithm.workers/evaluation_workers, local_worker(), etc. - **changed** — Add new python GcsClient binding #### Ray Libraries ##### Ray Data 💫 Enhancements: - Add better support for UDF returns from list of datetime objects (#46762) 🔨 Fixes: - Remove read task warning if size bytes not set in metadata (#46765) 📖 Documentation: - Fix read_tfrecords() docstring to display tfx-bsl tip (#46717) - Update Dataset.zip() docs (#46757) ##### Ray Train 🔨 Fixes: - Sort workers by node ID rather than by node IP (#46163) 🏗 Architecture refactoring: - Remove dead RayDatasetSpec (#46764) ##### RLlib 🎉 New Features: - Offline RL support on new API stack: - Initial design for Ray-Data based offline RL Algos (on new API stack). (#44969) - Add user-defined schemas for data loading. (#46738) - Make data pipeline better configurable and tuneable for users. (#46777) 💫 Enhancements: - Move DQN into the TargetNetworkAPI (and deprecate `RLModuleWithTargetNetworksInterface`). (#46752) 🔨 Fixes: - Numpy version fix: Rename all np.product usage to np.prod (#46317) 📖 Documentation: - Examples for new API stack: Add 2 (count-based) curiosity examples. (#46737) - Remove RLlib CLI from docs (soon to be deprecated and replaced by python API). (#46724) 🏗 Architecture refactoring: - Cleanup, rename, clarify: Algorithm.workers/evaluation_workers, local_worker(), etc.. (#46726) #### Ray Core 🏗 Architecture refactoring: - New python GcsClient binding (#46186) Many thanks to all those who contributed to this release! @KyleKoon, @ruisearch42, @rynewang, @sven1977, @saihaj, @aslonnie, @bveeramani, @akshay-anyscale, @kevin85421, @omatthew98, @anyscalesam, @MaxVanDijck, @justinvyu, @simonsays1980, @can-anyscale, @peytondmurray, @scottjlee ### 2.33.0 — Ray-2.33.0 - Date: 2024-07-25 - Version: 2.33.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.33.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.33.0 - **changed** — Add last exception to error message when GCS connection fails in ray.init() - **fixed** — Add object back to memory store when object recovery is skipped - **fixed** — Task status should start with PENDING_ARGS_AVAIL when retry - **fixed** — Fix ObjectFetchTimedOutError - **fixed** — Make working_dir support files created before 1980 - **fixed** — Allow full path in conda runtime env - **fixed** — Fix worker launch time formatting in state api - **deprecated** — Deprecate Dataset.get_internal_block_refs() - **added** — Add read API for reading Databricks table with Delta Sharing - **added** — Add support for objects to Arrow blocks - **changed** — Change offsets to int64 and change to LargeList for ArrowTensorArray - **changed** — Prevent from_pandas from combining input blocks - **changed** — Update Dataset.count() to avoid unnecessarily keeping BlockRefs in-memory - **changed** — Use Set to fix inefficient iteration over Arrow table columns - **changed** — Add AWS Error UNKNOWN to list of retried write errors - **changed** — Always print traceback for internal exceptions - **changed** — Allow unknown estimate of operator output bundles and ProgressBar totals - **changed** — Improve filesystem retry coverage - **fixed** — Replace lambda mutable default arguments in Ray Data - **changed** — Update run status and actor status for train runs #### Ray Libraries #### Ray Core 💫 Enhancements: - Add "last exception" to error message when GCS connection fails in ray.init() (#46516) 🔨 Fixes: - Add object back to memory store when object recovery is skipped (#46460) - Task status should start with PENDING_ARGS_AVAIL when retry (#46494) - Fix ObjectFetchTimedOutError ([#46562](https://github.com/ray-project/ray/pull/46562)) - Make working_dir support files created before 1980 ([#46634](https://github.com/ray-project/ray/pull/46634)) - Allow full path in conda runtime env. ([#45550](https://github.com/ray-project/ray/pull/45550)) - Fix worker launch time formatting in state api ([#43516](https://github.com/ray-project/ray/pull/43516)) - ##### Ray Data 🎉 New Features: - Deprecate Dataset.get_internal_block_refs() (#46455) - Add read API for reading Databricks table with Delta Sharing (#46072) - Add support for objects to Arrow blocks (#45272) 💫 Enhancements: - Change offsets to int64 and change to LargeList for ArrowTensorArray (#45352) - Prevent from_pandas from combining input blocks (#46363) - Update Dataset.count() to avoid unnecessarily keeping `BlockRef`s in-memory (#46369) - Use Set to fix inefficient iteration over Arrow table columns (#46541) - Add AWS Error UNKNOWN to list of retried write errors (#46646) - Always print traceback for internal exceptions (#46647) - Allow unknown estimate of operator output bundles and `ProgressBar` totals (#46601) - Improve filesystem retry coverage (#46685) 🔨 Fixes: - Replace lambda mutable default arguments (#46493) 📖 Documentation: - Auto-generate Dataset API documentation (#46557) - Update outdated ExecutionPlan docstring (#46638) ##### Ray Train 💫 Enhancements: - Update run status and actor status for train runs. (#46395) 🔨 Fixes: - Replace lambda default arguments (#46576) 📖 Documentation: - Add MNIST training using KubeRay doc page (#46123) - Add example of pre-training Llama model on Intel Gaudi (#45459) - Fix tensorflow example by using ScalingConfig (#46565) ##### Ray Tune 🔨 Fixes: - Replace lambda default arguments (#46596) ##### Ray Serve 🎉 New Features: - Fully deprecate `target_num_ongoing_requests_per_replica` and `max_concurrent_queries`, respectively replaced by `max_ongoing_requests` and `target_ongoing_requests` (#46392 and #46427) - Configure the task launched by the controller to build an application with Serve’s logging config (#46347) ##### RLlib 💫 Enhancements: - Moving sampling coordination for `batch_mode=complete_episodes` to `synchronous_parallel_sample`. (#46321) - Enable complex action spaces with stateful modules. (#46468) 🏗 Architecture refactoring: - Enable multi-learner setup for hybrid stack BC. (#46436) - Introduce Checkpointable API for RLlib components and subcomponents. (#46376) 🔨 Fixes: - Replace Mapping typehint with Dict: #46474 📖 Documentation: - More example scripts for new API stack: Two separate optimizers (w/ different learning rates). (#46540) and custom loss function. (#46445) ##### Dashboard 🔨 Fixes: - Task end time showing the incorrect time (#46439) - Events Table rows having really bad spacing (#46701) - UI bugs in the serve dashboard page (#46599) #### Thanks Many thanks to all those who contributed to this release! @alanwguo, @hongchaodeng, @anyscalesam, @brucebismarck, @bt2513, @woshiyyya, @terraflops1048576, @lorenzoritter, @omrishiv, @davidxia, @cchen777, @nono-Sang, @jackhumphries, @aslonnie, @JoshKarpel, @zjregee, @bveeramani, @khluu, @Superskyyy, @liuxsh9, @jjyao, @ruisearch42, @sven1977, @harborn, @saihaj, @zcin, @can-anyscale, @veekaybee, @chungen04, @WeichenXu123, @GeneDer, @sergey-serebryakov, @Bye-legumes, @scottjlee, @rynewang, @kevin85421, @cristianjd, @peytondmurray, @ _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.33.0]_ ### 2.32.0 — Ray-2.32.0 - Date: 2024-07-10 - Version: 2.32.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.32.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.32.0 - **added** — aDAG Developer Preview: Ray accelerated DAGs with Ray Core-like API and extensibility to pre-compile execution paths across pre-allocated resources - **added** — Support async callable classes in map_batches() - **added** — APPO on new API stack with EnvRunners - **added** — Added customizable refresh frequency for metrics on Ray Dashboard - **changed** — Optimize ServeController.get_app_config() - **changed** — Change default for max and target ongoing requests - **changed** — Integrate Ray Serve with Ray structured logging - **changed** — Allow configuring handle cache size and controller max concurrency - **changed** — Optimize DeploymentDetails.deployment_route_prefix_not_set() - **changed** — Allow env setup logger encoding - **changed** — ray list tasks filter state and name on GCS side - **changed** — Log ray version and ray commit during GCS start - **changed** — Upgraded Dashboard to MUIv5 and React 18 - **changed** — Make Tune trial ID available in EnvRunners and callbacks - **changed** — Add env- and agent_steps to custom evaluation function - **changed** — Remove default-metrics from Algorithm - **fixed** — Ensure InputDataBuffer doesn't free block references - **fixed** — MapOperator.num_active_tasks should exclude pending actors - **fixed** — Fix progress bars being displayed as partially completed in Jupyter notebooks - **fixed** — Decrement lineage ref count of an actor when the actor task return object reference is deleted #### Highlight: aDAG Developer Preview This is a new Ray Core specific feature called Ray accelerated DAGs (aDAGs). - aDAGs give you a Ray Core-like API but with extensibility to pre-compile execution paths across pre-allocated resources on a Ray Cluster to possible benefits for optimization on throughput and latency. Some practical examples include: - Up to 10x lower task execution time on single-node. - Native support for GPU-GPU communication, via NCCL. - This is still very early, but please reach out on #ray-core on Ray Slack to learn more! #### Ray Libraries ##### Ray Data 💫 Enhancements: - Support async callable classes in `map_batches()` (#46129) 🔨 Fixes: - Ensure `InputDataBuffer` doesn't free block references (#46191) - `MapOperator.num_active_tasks` should exclude pending actors (#46364) - Fix progress bars being displayed as partially completed in Jupyter notebooks (#46289) 📖 Documentation: - Fix docs: `read_api.py` docstring (#45690) - Correct API annotation for `tfrecords_datasource` (#46171) - Fix broken links in `README` and in `ray.data.Dataset` (#45345) ##### Ray Train 📖 Documentation: - Update PyTorch Data Ingestion User Guide (#45421) ##### Ray Serve 💫 Enhancements: - Optimize `ServeController.get_app_config()` (#45878) - Change default for max and target ongoing requests (#45943) - Integrate with Ray structured logging (#46215) - Allow configuring handle cache size and controller max concurrency (#46278) - Optimize `DeploymentDetails.deployment_route_prefix_not_set()` (#46305) ##### RLlib 🎉 New Features: - APPO on new API stack (w/ `EnvRunners`). (#46216) 💫 Enhancements: - Stability: APPO, SAC, and DQN activate multi-agent learning tests (#45542, #46299) - Make Tune trial ID available in `EnvRunners` (and callbacks). (#46294) - Add `env-` and `agent_steps` to custom evaluation function. (#45652) - Remove default-metrics from Algorithm (tune does NOT error anymore if any stop-metric is missing). (#46200) 🔨 Fixes: - Various bug fixes: #45542 📖 Documentation: - Example for new API stack: Offline RL (BC) training on single-agent, while evaluating w/ multi-agent setup. (#46251) - Example for new API stack: Custom RLModule with an LSTM. (#46276) #### Ray Core 🎉 New Features: - aDAG Developer Preview. 💫 Enhancements: - Allow env setup logger encoding (#46242) - ray list tasks filter state and name on GCS side (#46270) - Log ray version and ray commit during GCS start (#46341) 🔨 Fixes: - Decrement lineage ref count of an actor when the actor task return object reference is deleted (#46230) - Fix negative ALIVE actors metric and introduce IDLE state (#45718) - `psutil` process attr `num_fds` is not available on Windows (#46329) ##### Dashboard 🎉 New Features: - Added customizable refresh frequency for metrics on Ray Dashboard (#44037) 💫 Enhancements: - Upgraded to MUIv5 and React 18 (#45789) 🔨 Fixes: - Fix for multi-line log items breaking log viewer rendering (#46391) - Fix for UI inconsistency when a job submission creates more than one Ray job. (#46267) - Fix filtering by job id for tasks API not filtering correctly. (#45017) ##### Docs 🔨 Fixes: - Re-enabled automatic cross-reference link checking for Ray documentation, with Sphinx nitpicky mode (#46279) - Enforced naming conventions for public and private APIs to maintain accuracy, starting with Ray Data API documentation (#46261) 📖 Documentation: - Upgrade Python 3.12 support to alpha, marking the release of the Ray wheel to PyPI and conducting a sanity check of the most critical tests. #### Thanks Many thanks to all those who contributed to this release! @stephanie-wang, @MortalHappiness, @aslonnie, @ryanaoleary, @ _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.32.0]_ ### 2.31.0 — Ray-2.31.0 - Date: 2024-06-26 - Version: 2.31.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.31.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.31.0 - **fixed** — Fixed bug where preserve_order doesn't work with file reads - **added** — Added documentation for dataset.Schema - **added** — Add API for Ray Train run stats - **changed** — Missing stopping criterion should not error, just warn - **changed** — Optimize DeploymentStateManager.get_deployment_statuses - **fixed** — Fix logging error on passing traceback object into exc_info - **fixed** — Run __del__ even if constructor is still in-progress - **added** — IMPALA runs on the new API stack with EnvRunners and ConnectorV2s - **added** — SAC/DQN: Prioritized multi-agent episode replay buffer - **changed** — Automatically infer worker path when starting worker in container - **fixed** — On GCS restart, destroy not forget the unused workers, fixing PG leaks - **fixed** — Cancel lease requests before returning a PG bundle - **fixed** — Fix boost fiber stack overflow #### Ray Libraries ##### Ray Data 🔨 Fixes: - Fixed bug where `preserve_order` doesn’t work with file reads ([#46135](https://github.com/ray-project/ray/pull/46135)) 📖 Documentation: - Added documentation for `dataset.Schema` ([#46170](https://github.com/ray-project/ray/pull/46170)) ##### Ray Train 💫 Enhancements: - Add API for Ray Train run stats (#45711) ##### Ray Tune 💫 Enhancements: - Missing stopping criterion should not error (just warn). (#45613) 📖 Documentation: - Fix broken references in Ray Tune documentation (#45233) ##### Ray Serve **WARNING**: the following default values will change in Ray 2.32: - Default for `max_ongoing_requests` will change from 100 to 5. - Default for `target_ongoing_requests` will change from 1 to 2. 💫 Enhancements: - Optimize DeploymentStateManager.get_deployment_statuses ([#45872](https://github.com/ray-project/ray/pull/45872)) 🔨 Fixes: - Fix logging error on passing traceback object into exc_info ([#46105](https://github.com/ray-project/ray/pull/46105)) - Run __del__ even if constructor is still in-progress ([#45882](https://github.com/ray-project/ray/pull/45882)) - Spread replicas with custom resources in torch tune serve release test ([#46093](https://github.com/ray-project/ray/pull/46093)) - [1k release test] don't run replicas on head node ([#46130](https://github.com/ray-project/ray/pull/46130)) 📖 Documentation: - Remove todo since issue is fixed ([#45941](https://github.com/ray-project/ray/pull/45941)) ##### RLlib 🎉 New Features: - IMPALA runs on the new API stack (with EnvRunners and ConnectorV2s). ([#42085](https://github.com/ray-project/ray/pull/42085)) - SAC/DQN: Prioritized multi-agent episode replay buffer. ([#45576](https://github.com/ray-project/ray/pull/45576)[)](https://github.com/ray-project/ray/commit/a7aa5e4c7bc4654b5ce710bf3551e3775470b57b) 💫 Enhancements: - New API stack stability: Add systematic CI learning tests for all possible combinations of: [PPO|IMPALA] + [1CPU|2CPU|1GPU|2GPU] + [single-agent|multi-agent]. ([#46162](https://github.com/ray-project/ray/pull/46162), [#46161](https://github.com/ray-project/ray/pull/46161)) 📖 Documentation: - New API stack: Example script for action masking [(](https://github.com/ray-project/ray/commit/9d661677f942ccd7dfbbf2c2838e5dad9f50d12e)[#46146](https://github.com/ray-project/ray/pull/46146)) - New API stack: PyFlight example script cleanup ([#45956](https://github.com/ray-project/ray/pull/45956)[)](https://github.com/ray-project/ray/commit/af45a8999b5d9ca09054de29c2bdd0b5b02184b5) - Old API stack: Enhanced ONNX example (+LSTM). ([#43592](https://github.com/ray-project/ray/pull/43592)[)](https://github.com/ray-project/ray/commit/d15204f5a0d5f2fa70935efa4935a7a61d8aafd1) #### Ray Core and Ray Clusters ##### Ray Core 💫 Enhancements: - [runtime-env] automatically infer worker path when starting worker in container ([#42304](https://github.com/ray-project/ray/pull/42304)) 🔨 Fixes: - On GCS restart, destroy not forget the unused workers. Fixing PG leaks. (#45854) - Cancel lease requests before returning a PG bundle ([#45919](https://github.com/ray-project/ray/pull/45919)) - Fix boost fiber stack overflow (#46133) #### Thanks Many thanks to all those who contributed to this release! @jjyao, @kevin85421, @vincent-pli, @khluu, @simonsays1980, @sven1977, @rynewang, @can-anyscale, @richardsliu, @jackhumphries, @alexeykudinkin, @bveeramani, @ruisearch42, @shrekris-anyscale, @stephanie-wang, @matthewdeng, @zcin, @hongchaodeng, @ryanaoleary, @liuxsh9, @GeneDer, @aslonnie, @peytondmurray, @Bye-legumes, @woshiyyya, @scottjlee, @JoshKarpel ### 2.30.0 — Ray-2.30.0 - Date: 2024-06-20 - Version: 2.30.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.30.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.30.0 - **changed** — Improve fractional CPU/GPU formatting - **changed** — Use sampled fragments to estimate Parquet reader batch size - **changed** — Refactor ParquetDatasource and metadata fetching logic - **changed** — Refactor planner.py - **changed** — Change the behavior of a missing stopping criterion metric to warn instead of raising an error - **added** — Create internal request id to track request objects - **changed** — Enhance env-rendering callback - **fixed** — Fix EMA stats without window that would lead to infinite list memory leak in MetricsLogger API - **added** — Alpha release of job level logging configuration allowing users to configure user logging in logfmt format with logging context - **changed** — Integrate amdsmi in AMDAcceleratorManager - **fixed** — Fix the C++ GcsClient Del not respecting del_by_prefix - **fixed** — Fix exit handling of FiberState threads - **changed** — Parse out json logs in Dashboard #### Ray Libraries ##### Ray Data 💫 Enhancements: - Improve fractional CPU/GPU formatting (#45673) - Use sampled fragments to estimate Parquet reader batch size (#45749) - Refactoring ParquetDatasource and metadata fetching logic (#45728, #45727, #45733, #45734, #45767) - Refactor planner.py (#45706) ##### Ray Tune 💫 Enhancements: - Change the behavior of a missing stopping criterion metric to warn instead of raising an error. This enables the use case of reporting different sets of metrics on different iterations (ex: a separate set of training and validation metrics). ([#45613](https://github.com/ray-project/ray/pull/45613)) ##### Ray Serve 💫 Enhancements: - Create internal request id to track request objects ([#45761](https://github.com/ray-project/ray/pull/45761)) ##### RLLib 💫 Enhancements: - Stability: [DreamerV3 weekly release test](https://github.com/ray-project/ray/commit/4adb78b2bf3c968f88f72ae9064189b846833230) ([#45654](https://github.com/ray-project/ray/pull/45654)); [Add "official" benchmark script for Atari PPO benchmarks](https://github.com/ray-project/ray/commit/d49f15b1112e67d15a80d696249f587ea7b95b57). ([#45697](https://github.com/ray-project/ray/pull/45697)) - Enhance env-rendering callback (#45682) 🔨 Fixes: - Bug fix in new MetricsLogger API: [EMA stats w/o window would lead to infinite l](https://github.com/ray-project/ray/commit/cbb1634a23ff4b59090f43dd853cf437e19fc0c8)ist mem-leak. ([#45752](https://github.com/ray-project/ray/pull/45752)) - Various other bug fixes: ([#45819](https://github.com/ray-project/ray/pull/45819), [#45820](https://github.com/ray-project/ray/pull/45820), #[45683](https://github.com/ray-project/ray/pull/45683), [#45651](https://github.com/ray-project/ray/pull/45651), [#45753](https://github.com/ray-project/ray/pull/45753)) 📖 Documentation: - Re-do `examples` overview page (new API stack): [#45382](https://github.com/ray-project/ray/pull/45382) - PyFlyt QuadX WayPoints example [#44758](https://github.com/ray-project/ray/pull/44758), [#45956](https://github.com/ray-project/ray/pull/45956) - RLModule inference on new API stack ([#45831](https://github.com/ray-project/ray/pull/45831), [#45845](https://github.com/ray-project/ray/pull/45845)) - How to resume a tune.Tuner.fit() experiment from checkpoint. ([#45681](https://github.com/ray-project/ray/pull/45681)) - Custom RLModule (tiny CNN): [#45774](https://github.com/ray-project/ray/pull/45774) - Connector examples docstrings ([#45864](https://github.com/ray-project/ray/pull/45864)) - Old API stack examples: [#43592](https://github.com/ray-project/ray/pull/43592), [#45829](https://github.com/ray-project/ray/pull/45829) ##### Ray Core 🎉 New Features: - Alpha release of job level [logging configuration](https://docs.ray.io/en/master/ray-core/api/doc/ray.LoggingConfig.html#ray.LoggingConfig): users can now config the user logging to be logfmt format with logging context attached. (#45344) 💫 Enhancements: - Integrate amdsmi in AMDAcceleratorManager (#44572) 🔨 Fixes: - Fix the C++ GcsClient Del not respecting del_by_prefix (#45604) - Fix exit handling of FiberState threads (#45834) ##### Dashboard 💫 Enhancements: - Parse out json logs (#45853) Many thanks to all those who contributed to this release: @liuxsh9, @peytondmurray, @pcmoritz, @GeneDer, @saihaj, @khluu, @aslonnie, @yucai, @vickytsang, @can-anyscale, @bthananjeyan, @raulchen, @hongchaodeng, @x13n, @simonsays1980, @peterghaddad, @kevin85421, @rynewang, @angelinalg, @jjyao, @BenWilson2, @jackhumphries, @zcin, @chris-ray-zhang, @c21, @shrekris-anyscale, @alanwguo, @stephanie-wang, @Bye-legumes, @sven1977, @WeichenXu123, @bveeramani, @nikitavemuri ### 2.24.0 — Ray-2.24.0 - Date: 2024-06-06 - Version: 2.24.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.24.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.24.0 - **added** — Allow user to configure timeout for actor pool - **added** — Add override_num_blocks to from_pandas and perform auto-partition - **changed** — Upgrade Arrow version to 16 in CI - **changed** — Record more telemetry for newly added datasources - **fixed** — Avoid pickling LanceFragment when creating read tasks for Lance - **deprecated** — Deprecate TUNE_RESULT_DIR, RAY_AIR_LOCAL_CACHE_DIR, and local_dir - **changed** — Improve excessive syncing warning - **changed** — Clean up Serve proxy files - **fixed** — DreamerV3 on tf bug fix to run again with tf==2.11.1 - **added** — Add support for multi-agent off-policy algorithms (DQN and SAC) in the new API - **added** — Config option for APPO/IMPALA to change number of GPU-loader threads - **fixed** — Various MetricsLogger bug fixes - **fixed** — Fix worker crash when getting actor name from runtime context - **fixed** — Log dedup should not dedup number only lines - **changed** — Improve node death observability - **added** — Ray c++ backend structured logging - **fixed** — Move Job package uploading to another thread to unblock the event loop - **changed** — Clarify that num_rows_per_file is not strict #### Ray Libraries ##### Ray Data 🎉 New Features: - Allow user to configure timeout for actor pool (#45508) - Add override_num_blocks to from_pandas and perform auto-partition (#44937) - Upgrade Arrow version to 16 in CI (#45565) 💫 Enhancements: - Clarify that num_rows_per_file isn't strict (#45529) - Record more telemetry for newly added datasources (#45647) - Avoid pickling LanceFragment when creating read tasks for Lance (#45392) ##### Ray Train 📖 Documentation: - [HPU] Add example of Stable Diffusion fine-tuning and serving on Intel Gaudi (#45217) - [HPU] Add example of Llama-2 fine-tuning on Intel Gaudi (#44667) ##### Ray Tune 🏗 Architecture refactoring: - Improve excessive syncing warning and deprecate TUNE_RESULT_DIR, RAY_AIR_LOCAL_CACHE_DIR, local_dir (#45210) ##### Ray Serve 💫 Enhancements: - Clean up Serve proxy files ([#45486](https://github.com/ray-project/ray/pull/45486)) 📖 Documentation: - vllm example to serve llm models ([#45430](https://github.com/ray-project/ray/pull/45430)) ##### RLLib 💫 Enhancements: - DreamerV3 on tf: Bug fix, so it can run again with tf==2.11.1 (2.11.0 is not available anymore) (#45419); Added weekly release test for DreamerV3. - Added support for multi-agent off-policy algorithms (DQN and SAC) in the new (#45182) - Config option for APPO/IMPALA to change number of GPU-loader threads (#45467) 🔨 Fixes: - Various MetricsLogger bug fixes (#45543, #45585, #45575) - Other fixes: #45588, #45617, #45517, #45465 📖 Documentation: - Example script for new API stack: How-to restore 1 of n agents from a checkpoint. (#45462) - Example script for new API stack: Autoregressive action module. #45525 ##### Ray Core 💫 Enhancements: - Improve node death observability (#45320, #45357, #45533, #45644, #45497) - Ray c++ backend structured logging (#44468) 🔨 Fixes: - Fix worker crash when getting actor name from runtime context (#45194) - log dedup should not dedup number only lines (#45385) 📖 Documentation: - Improve doc for `--object-store-memory` to describe how the default value is set ([#45301](https://github.com/ray-project/ray/pull/45301)) ##### Dashboard 🔨 Fixes: - Move Job package uploading to another thread to unblock the event loop. ([#45282](https://github.com/ray-project/ray/pull/45282)) Many thanks to all those who contributed to this release: @maxliuofficial, @simonsays1980, @GeneDer, @dudeperf3ct, @khluu, @justinvyu, @andrewsykim, @Catch-Bull, @zcin, @bveeramani, @rynewang, @angelinalg, @matthewdeng, @jjyao, @kira-lin, @harborn, @hongchaodeng, @peytondmurray, @aslonnie, @timkpaine, @982945902, @maxpumperla, @stephanie-wang, @ruisearch42, @alanwguo, @can-anyscale, @c21, @Atry, @KamenShah, @sven1977, @raulchen ### 2.23.0 — Ray-2.23.0 - Date: 2024-05-22 - Version: 2.23.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.23.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.23.0 - **added** — Add support for using GPUs with map_groups - **added** — Add support for using actors with map_groups - **changed** — Refine exception handling from arrow data conversion - **fixed** — Fix Ray databricks UC reader with dynamic Databricks notebook scope token - **fixed** — Fix bug where you can't return objects and array from UDF - **fixed** — Fix bug where map_groups triggers execution during input validation - **fixed** — Fix PB2 scheduler error resulting from trying to sort by Trial objects - **fixed** — Log application unhealthy errors at error level instead of warning level - **changed** — Examples and tuned_examples learning test for new API stack are now self-executable and support WandB - **fixed** — Fix result dict spam with duplicate and deprecated keys in top level - **changed** — Add runtime env started logs to job driver - **changed** — ray.util.collective support torch.bfloat16 - **changed** — Better propagate node death information - **fixed** — Fix worker process leaks after job finishes #### Ray Libraries ##### Ray Data 🎉 New Features: - Add support for using GPUs with map_groups (#45305) - Add support for using actors with map_groups (#45310) 💫 Enhancements: - Refine exception handling from arrow data conversion (#45294) 🔨 Fixes: - Fix Ray databricks UC reader with dynamic Databricks notebook scope token (#45153) - Fix bug where you can't return objects and array from UDF (#45287 ) - Fix bug where map_groups triggers execution during input validation (#45314) ##### Ray Tune 🔨 Fixes: - [tune] Fix PB2 scheduler error resulting from trying to sort by Trial objects (#45161) ##### Ray Serve 🔨 Fixes: - Log application unhealthy errors at error level instead of warning level ([#45211](https://github.com/ray-project/ray/pull/45211)) ##### RLLib 💫 Enhancements: - Examples and `tuned_examples` learning test for new API stack are now “self-executable” (don’t require a third-party script anymore to run them). + WandB support. ([#45023](https://github.com/ray-project/ray/pull/45023)) 🔨 Fixes: - Fix result dict “spam” (duplicate, deprecated keys, e.g. “sampler_results” dumped into top level). ([#45330](https://github.com/ray-project/ray/pull/45330)) 📖 Documentation: - Add example for training with fractional GPUs on new API stack. ([#45379](https://github.com/ray-project/ray/pull/45379)) - Cleanup examples folder and remove deprecated sub directories. ([#45327](https://github.com/ray-project/ray/pull/45327)) ##### Ray Core 💫 Enhancements: - [Logs] Add runtime env started logs to job driver ([#45255](https://github.com/ray-project/ray/pull/45255)) - `ray.util.collective` support `torch.bfloat16` ([#39845](https://github.com/ray-project/ray/pull/39845)) - [Core] Better propagate node death information ([#45128](https://github.com/ray-project/ray/pull/45128)) 🔨 Fixes: - [Core] Fix worker process leaks after job finishes ([#44214](https://github.com/ray-project/ray/pull/44214)) Many thanks to all those who contributed to this release: @hongchaodeng, @khluu, @antoni-jamiolkowski, @ameroyer, @bveeramani, @can-anyscale, @WeichenXu123, @peytondmurray, @jackhumphries, @kevin85421, @jjyao, @robcaulk, @rynewang, @scottsun94, @swang, @GeneDer, @zcin, @ruisearch42, @aslonnie, @angelinalg, @raulchen, @ArthurBook, @sven1977, @wuxibin89 ### 2.22.0 — Ray-2.22.0 - Date: 2024-05-14 - Version: 2.22.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.22.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.22.0 - **added** — Add function to dynamically generate ray_remote_args for Map APIs - **added** — Allow manually setting resource limits for training jobs - **changed** — Introduce abstract interface for data autoscaling - **changed** — Add debugging info for SplitCoordinator - **fixed** — Don't show AllToAllOperator progress bar if the disable flag is set - **fixed** — Don't load Arrow PyExtensionType by default - **fixed** — Don't raise batch size error if num_gpus=0 - **changed** — Update RayTrainReportCallback to only save checkpoints on rank 0 for XGBoost and LightGBM - **fixed** — Fix the cpu percentage metrics for dashboard process - **changed** — Improve log viewer so line numbers do not get selected when copying text - **changed** — Improve log viewer to avoid unnecessary re-rendering which causes text selection to clear #### Ray Libraries ##### Ray Data 🎉 New Features: - Add function to dynamically generate `ray_remote_args` for Map APIs (#45143) - Allow manually setting resource limits for training jobs (#45188) 💫 Enhancements: - Introduce abstract interface for data autoscaling (#45002) - Add debugging info for `SplitCoordinator` (#45226) 🔨 Fixes: - Don’t show `AllToAllOperator` progress bar if the disable flag is set (#45136) - Don't load Arrow `PyExtensionType` by default (#45084) - Don't raise batch size error if `num_gpus=0` (#45202) ##### Ray Train 💫 Enhancements: - [XGBoost][LightGBM] Update RayTrainReportCallback to only save checkpoints on rank 0 (#45083) #### Ray Core 🔨 Fixes: - Fix the cpu percentage metrics for dashboard process (#45124) ##### Dashboard 💫 Enhancements: - Improvements to log viewer so line numbers do not get selected when copying text. - Improvements to the log viewer to avoid unnecessary re-rendering which causes text selection to clear. Many thanks to all those who contributed to this release: @justinvyu, @simonsays1980, @chris-ray-zhang, @kevin85421, @angelinalg, @rynewang, @brycehuang30, @alanwguo, @jjyao, @shaikhismail, @khluu, @can-anyscale, @bveeramani, @jrosti, @WeichenXu123, @MortalHappiness, @raulchen, @scottjlee, @ruisearch42, @aslonnie, @alexeykudinkin ### 2.21.0 — Ray-2.21.0 - Date: 2024-05-08 - Version: 2.21.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.21.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.21.0 - **added** — Add read_lance API to read Lance Dataset - **fixed** — Retry RaySystemError application errors - **removed** — Remove deprecated ray.air.callbacks modules - **changed** — Allow methods to pass type @serve.batch type hint - **changed** — Allow configuring Serve control loop interval - **fixed** — Fix bug with controller failing to recover for autoscaling deployments - **fixed** — Fix control+c after serve run doesn't shutdown serve components - **fixed** — Fix lightweight update max ongoing requests - **added** — New MetricsLogger API now fully functional on the new API stack, working inside Learner classes and loss functions - **changed** — Rename WorkerSet to EnvRunnerGroup, DEFAULT_POLICY_ID to DEFAULT_MODULE_ID, config.rollouts() to config.env_runners() - **changed** — Change behavior of EnvRunnerGroup.foreach_worker methods to mark_healthy=True and healthy_only=True as new defaults - **fixed** — Fix get_state() and from_state() methods in SingleAgent and MultiAgentEpisodes - **fixed** — Fix torch global_norm clipping overflow problem - **fixed** — Fix ray.init(logging_format) argument is ignored - **fixed** — Handle unserializable user exception - **fixed** — Fix dashboard process event loop blocking issues - **fixed** — Fix Nodes page sorting not working correctly - **fixed** — Add back actors per page UI control in the actors page #### Ray Libraries ##### Ray Data 🎉 New features: - Add `read_lance` API to read Lance Dataset (#45106) 🔨 Fixes: - Retry RaySystemError application errors (#45079) 📖 Documentation: - Fix broken references in data documentation (#44956) ##### Ray Train 📖 Documentation: - Fix broken links in Train documentation (#44953) ##### Ray Tune 📖 Documentation: - Update Hugging Face example to add reference (#42771) 🏗 Architecture refactoring: - Remove deprecated `ray.air.callbacks` modules (#45104) ##### Ray Serve 💫 Enhancements: - Allow methods to pass type @serve.batch type hint (#45004) - Allow configuring Serve control loop interval (#45063) 🔨 Fixes: - Fix bug with controller failing to recover for autoscaling deployments (#45118) - Fix control+c after serve run doesn't shutdown serve components (#45087) - Fix lightweight update max ongoing requests (#45006) ##### RLlib 🎉 New Features: - New MetricsLogger API now fully functional on the new API stack (working now also inside Learner classes, i.e. loss functions). ([#44995](https://github.com/ray-project/ray/pull/44995), [#45109](https://github.com/ray-project/ray/pull/45109)) 💫 Enhancements: - Renamings and cleanups (toward new API stack and more consistent naming schemata): WorkerSet -> EnvRunnerGroup, DEFAULT_POLICY_ID -> DEFAULT_MODULE_ID, config.rollouts() -> config.env_runners(), etc.. ([#45022](https://github.com/ray-project/ray/pull/45022), [#44920](https://github.com/ray-project/ray/pull/44920)) - Changed behavior of `EnvRunnerGroup.foreach_worker…` methods to new defaults: `mark_healthy=True` (used to be False) and `healthy_only=True` (used to be False). ([#44993](https://github.com/ray-project/ray/pull/44993)) - Fix `get_state()/from_state()` methods in SingleAgent- and MultiAgentEpisodes. ([#45012](https://github.com/ray-project/ray/pull/45012)) 🔨 Fixes: - Bug fix for (torch) global_norm clipping overflow problem: ([#45055](https://github.com/ray-project/ray/pull/45055)) - Various bug- and test case fixes: [#45030](https://github.com/ray-project/ray/pull/45030), [#45031](https://github.com/ray-project/ray/pull/45031), [#45070](https://github.com/ray-project/ray/pull/45070), [#45053](https://github.com/ray-project/ray/pull/45053), [#45110](https://github.com/ray-project/ray/pull/45110), [#45111](https://github.com/ray-project/ray/pull/45111) 📖 Documentation: - Example scripts using the MetricsLogger for env rendering and recording w/ WandB: [#45073](https://github.com/ray-project/ray/pull/45073), [#45107](https://github.com/ray-project/ray/pull/45107) #### Ray Core 🔨 Fixes: - Fix `ray.init(logging_format)` argument is ignored (#45037) - Handle unserializable user exception (#44878) - Fix dashboard process event loop blocking issues (#45048, #45047) ##### Dashboard 🔨 Fixes: - Fix Nodes page sorting not working correctly. - Add back “actors per page” UI control in the actors page. Many thanks to all those who contributed to this release: @rynewang, @can-anyscale, @scottsun94, @bveeramani, @ceddy4395, @GeneDer, @zcin, @JoshKarpel, @nikitavemuri, @stephanie-wang, @jackhumphries, @matthewdeng, @yash97, @simonsays1980, @peytondmurray, @evalaiyc98, @c21, @alanwguo, @shrekris-anyscale, @kevin85421, @hongchaodeng, @sven1977, @st--, @khluu ### 2.20.0 — Ray-2.20.0 - Date: 2024-05-01 - Version: 2.20.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.20.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.20.0 - **changed** — Deduplicate repeated schema during ParquetDatasource metadata prefetching - **changed** — Update map_groups implementation to better handle large outputs - **deprecated** — Deprecate prefetch_batches argument of iter_rows and change default value - **changed** — Add default behavior to false for creating directories on S3 writes - **changed** — Make internal UDF names more descriptive - **changed** — Make name a required argument for AggregateFn - **changed** — Setup XGBoost CommunicatorContext automatically - **added** — Track Train Run Info with TrainStateActor - **changed** — Remove trial table when running Ray Train in a Jupyter notebook - **changed** — Clean up temporary checkpoint directories for class Trainables - **changed** — Make handle push metric interval configurable with environment variable RAY_SERVE_HANDLE_METRIC_PUSH_INTERVAL_S - **changed** — Improve performance of developer API serve.get_app_handle - **fixed** — Fix memory leak in handles for autoscaling deployments when RAY_SERVE_COLLECT_AUTOSCALING_METRICS_ON_HANDLE=1 - **added** — Introduce MetricsLogger, a unified API for logging custom metrics and stats in all of RLlib's components - **added** — Introduce inference-only (slim) mode for RLModules that run inside an EnvRunner - **added** — Add MultiAgentEpisodeReplayBuffer for new API stack in preparation for multi-agent support - **changed** — Clean up AlgorithmConfig and rename properties and methods for better consistency and transparency - **changed** — Report GCS internal pubsub buffer metrics and cap message size - **fixed** — Fix task submission never returning when network partition happens - **fixed** — Fix incorrect use of SSH port forward option #### Ray Libraries ##### Ray Data 💫 Enhancements: - Dedupe repeated schema during `ParquetDatasource` metadata prefetching (#44750) - Update `map_groups` implementation to better handle large outputs (#44862) - Deprecate `prefetch_batches` arg of `iter_rows` and change default value (#44982) - Adding in default behavior to false for creating dirs on s3 writes (#44972) - Make internal UDF names more descriptive (#44985) - Make `name` a required argument for `AggregateFn` (#44880) 📖 Documentation: - Add key concepts to and revise "Data Internals" page (#44751) ##### Ray Train 💫 Enhancements: - Setup XGBoost `CommunicatorContext` automatically (#44883) - Track Train Run Info with `TrainStateActor` (#44585) 📖 Documentation: - Add documentation for `accelerator_type` (#44882) - Update Ray Train example titles (#44369) ##### Ray Tune 💫 Enhancements: - Remove trial table when running Ray Train in a Jupyter notebook (#44858) - Clean up temporary checkpoint directories for class Trainables (ex: RLlib) (#44366) 📖 Documentation: - Fix minor doc format issues (#44865) - Remove outdated ScalingConfig references (#44918) ##### Ray Serve 💫 Enhancements: - Handle push metric interval is now configurable with environment variable RAY_SERVE_HANDLE_METRIC_PUSH_INTERVAL_S (#32920) - Improve performance of developer API serve.get_app_handle (#44812) 🔨 Fixes: - Fix memory leak in handles for autoscaling deployments (the leak happens when - RAY_SERVE_COLLECT_AUTOSCALING_METRICS_ON_HANDLE=1) (#44877) ##### RLlib 🎉 New Features: - Introduce `MetricsLogger`, a unified API for users of RLlib to log custom metrics and stats in all of RLlib’s components (Algorithm, EnvRunners, and Learners). Rolled out for new API stack for Algorithm (`training_step`) and EnvRunners (custom callbacks). `Learner` (custom loss functions) support in progress. [#44888](https://github.com/ray-project/ray/pull/44888), [#44442](https://github.com/ray-project/ray/pull/44442) - Introduce “inference-only” (slim) mode for RLModules that run inside an EnvRunner (and thus don’t require value-functions or target networks): [#44797](https://github.com/ray-project/ray/pull/44797) 💫 Enhancements: - MultiAgentEpisodeReplayBuffer for new API stack (preparation for multi-agent support of SAC and DQN): [#44450](https://github.com/ray-project/ray/pull/44450) - AlgorithmConfig cleanup and renaming of properties and methods for better consistency/transparency: [#44896](https://github.com/ray-project/ray/pull/44896) 🔨 Fixes: - Various minor bug fixes: [#44989](https://github.com/ray-project/ray/pull/44989), [#44988](https://github.com/ray-project/ray/pull/44988), [#44891](https://github.com/ray-project/ray/pull/44891), [#44898](https://github.com/ray-project/ray/pull/44898), [#44868](https://github.com/ray-project/ray/pull/44868), [#44867](https://github.com/ray-project/ray/pull/44867), [#44845](https://github.com/ray-project/ray/pull/44845) ##### Ray Core and Ray Clusters 💫 Enhancements: - Report GCS internal pubsub buffer metrics and cap message size (#44749) 🔨 Fixes: - Fix task submission never return when network partition happens (#44692) - Fix incorrect use of ssh port forward option. (#44973) - Make sure dashboard will exit if grpc server fails (#44928) - Make sure dashboard agent will exit if grpc server fails (#44899) Thanks @can-anyscale, @hongchaodeng, @zcin, @marwan116, @khluu, @bewestphal, @scottjlee, @andrewsykim, @anyscalesam, @MortalHappiness, @justinvyu, @JoshKarpel, @woshiyyya, @rynewang, @Abirdcfly, @omatthew98, @sven1977, @marcelocarmona, @rueian, @mattip, @angelinalg, @aslonnie, @matthewdeng, @abizjakpro, @simonsays1980, @jjyao, @terraflops1048576, @hongpeng-guo, @stephanie-wang, @bw-matthew, @bveeramani, @ruisearch42, @ke _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.20.0]_ ### 2.12.0 — Ray-2.12.0 - Date: 2024-04-25 - Version: 2.12.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.12.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.12.0 - **added** — Store Ray Data logs in special subdirectory - **added** — Add in `local_read` option to `from_torch` - **fixed** — Fix the config to disable progress bar - **fixed** — Disable gathering the full state dict in `RayFSDPStrategy` for `lightning>2.1` - **changed** — Remove spammy log for "new output engine" - **fixed** — Fix getting attributes on stdout during Serve logging redirect - **added** — Support of images and video logging in WandB - **changed** — Better support and separation-of-concerns for `model_config_dict` in new API stack - **added** — Example script to pre-train an `RLModule` in single-agent fashion, then bring checkpoint into multi-agent setup and continue training - **changed** — More example scripts got translated from the old- to the new API stack including Curriculum learning and custom-gym-env - **fixed** — Fix GetAllJobInfo `is_running_tasks` is not returning the correct value when driver starts ray #### Ray Libraries ##### Ray Data 🎉 New Features: - Store Ray Data logs in special subdirectory (#44743) 💫 Enhancements: - Add in `local_read` option to `from_torch` (#44752) 🔨 Fixes: - Fix the config to disable progress bar (#44342) 📖 Documentation: - Clarify deprecated Datasource docstrings (#44790) ##### Ray Train 🔨 Fixes: - Disable gathering the full state dict in `RayFSDPStrategy` for `lightning>2.1` (#44569) ##### Ray Tune 💫 Enhancements: - Remove spammy log for "new output engine" (#44824) - Enable isort (#44693) ##### Ray Serve 🔨 Fixes: - [Serve] fix getting attributes on stdout during Serve logging redirect ([#44787](https://github.com/ray-project/ray/pull/44787)) ##### RLlib 🎉 New Features: - Support of images and video logging in WandB (env rendering example script for the new API stack coming up). ([#43356](https://github.com/ray-project/ray/pull/43356)) 💫 Enhancements: - Better support and separation-of-concerns for `model_config_dict` in new API stack. ([#44263](https://github.com/ray-project/ray/pull/44263)) - Added example script to pre-train an `RLModule` in single-agent fashion, then bring checkpoint into multi-agent setup and continue training. ([#44674](https://github.com/ray-project/ray/pull/44674)) - More `examples` scripts got translated from the old- to the new API stack: Curriculum learning, custom-gym-env, etc..: ([#44706](https://github.com/ray-project/ray/pull/44706), [#44707](https://github.com/ray-project/ray/pull/44707), [#44735](https://github.com/ray-project/ray/pull/44735), [#44841](https://github.com/ray-project/ray/pull/44841)) ##### Ray Core and Ray Clusters 🔨 Fixes: - Fix GetAllJobInfo `is_running_tasks` is not returning the correct value when driver starts ray (#44459) #### Thanks Many thanks to all those who contributed to this release! @can-anyscale, @hongpeng-guo, @sven1977, @zcin, @shrekris-anyscale, @liuxsh9, @jackhumphries, @GeneDer, @woshiyyya, @simonsays1980, @omatthew98, @andrewsykim, @n30111, @architkulkarni, @bveeramani, @aslonnie, @alexeykudinkin, @WeichenXu123, @rynewang, @matthewdeng, @angelinalg, @c21 ### 2.11.0 — Ray-2.11.0 - Date: 2024-04-17 - Version: 2.11.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.11.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.11.0 - **added** — Support reading Avro files with ray.data.read_avro - **added** — Add experimental support for AWS Trainium (Neuron) - **added** — Add experimental support for Intel HPU - **changed** — Pin ipywidgets==7.7.2 to enable Data progress bars in VSCode Web - **changed** — Change log level for ignored exceptions - **changed** — Change Parquet encoding ratio lower bound from 2 to 1 - **changed** — Add support for new style lightning import - **fixed** — Fix throughput time calculations for metrics - **fixed** — Fix nested ragged numpy.ndarray - **fixed** — Fix Ray debugger incompatibility caused by trimmed error stack trace - **fixed** — Fix ScalingConfig(accelerator_type) to request an appropriate resource amount - **fixed** — Fix maximum recursion issue when serializing exceptions - **fixed** — Remove base config deepcopy when initializing the trainer actor - **fixed** — Dynamically-created applications will no longer be deleted when a config is PUT via the REST API - **fixed** — Fix _to_object_ref memory leak - **fixed** — Fix multi-GPU and multi-agent support on the new API stack - **removed** — Remove deprecated BatchPredictor - **removed** — Remove scikit-optimize search algorithm - **removed** — Remove batch_queue_cls parameter from the @serve.batch decorator - **deprecated** — Log a deprecation warning for local_dir and related environment variables #### Release Highlights - [data] Support reading Avro files with `ray.data.read_avro` - [train] Added experimental support for AWS Trainium (Neuron) and Intel HPU. #### Ray Libraries ##### Ray Data 🎉 New Features: - Support reading Avro files with `ray.data.read_avro` (#43663) 💫 Enhancements: - Pin `ipywidgets==7.7.2` to enable Data progress bars in VSCode Web (#44398) - Change log level for ignored exceptions (#44408) 🔨 Fixes: - Change Parquet encoding ratio lower bound from 2 to 1 (#44470) - Fix throughput time calculations for metrics (#44138) - Fix nested ragged `numpy.ndarray` (#44236) - Fix Ray debugger incompatibility caused by trimmed error stack trace (#44496) 📖 Documentation: - Update "Data Loading and Preprocessing" doc (#44165) - Move imports into `TFPRedictor` in batch inference example (#44434) ##### Ray Train 🎉 New Features: - Add experimental support for AWS Trainium (Neuron) (#39130) - Add experimental support for Intel HPU (#43343) 💫 Enhancements: - Log a deprecation warning for local_dir and related environment variables (#44029) - Enforce xgboost>=1.7 for XGBoostTrainer usage (#44269) 🔨 Fixes: - Fix ScalingConfig(accelerator_type) to request an appropriate resource amount (#44225) - Fix maximum recursion issue when serializing exceptions (#43952) - Remove base config deepcopy when initializing the trainer actor (#44611) 🏗 Architecture refactoring: - Remove deprecated `BatchPredictor` (#43934) ##### Ray Tune 💫 Enhancements: - Add support for new style lightning import (#44339) - Log a deprecation warning for local_dir and related environment variables (#44029) 🏗 Architecture refactoring: - Remove scikit-optimize search algorithm (#43969) ##### Ray Serve 🔨 Fixes: - Dynamically-created applications will no longer be deleted when a config is PUT via the REST API ([#44476](https://github.com/ray-project/ray/pull/44476)). - Fix `_to_object_ref` memory leak ([#43763](https://github.com/ray-project/ray/pull/43763)) - Log warning to reconfigure `max_ongoing_requests` if `max_batch_size` is less than `max_ongoing_requests` ([#43840](https://github.com/ray-project/ray/pull/43840)) - Deployment fails to start with `ModuleNotFoundError` in Ray 3.10 ([#44329](https://github.com/ray-project/ray/issues/44329)) - This was fixed by reverting the original core changes on the `sys.path` behavior. Revert "[core] If there's working_dir, don't set _py_driver_sys_path." ([#44435](https://github.com/ray-project/ray/pull/44435)) - The `batch_queue_cls` parameter is removed from the `@serve.batch` decorator ([#43935](https://github.com/ray-project/ray/pull/43935)) ##### RLlib 🎉 New Features: - New API stack: **DQN Rainbow** is now available for single-agent ([#43196](https://github.com/ray-project/ray/pull/43196), [#43198](https://github.com/ray-project/ray/pull/43198), [#43199](https://github.com/ray-project/ray/pull/43199)) - **`PrioritizedEpisodeReplayBuffer`** is available for **off-policy learning using the EnvRunner API** (`SingleAgentEnvRunner`) and supports random n-step sampling ([#42832](https://github.com/ray-project/ray/pull/42832), [#43258](https://github.com/ray-project/ray/pull/43258), [#43458](https://github.com/ray-project/ray/pull/43458), [#43496](https://github.com/ray-project/ray/pull/43496), [#44262](https://github.com/ray-project/ray/pull/44262)) 💫 Enhancements: - **Restructured `examples/` folder**; started moving example scripts to the new API stack ([#44559](https://github.com/ray-project/ray/pull/44559), [#44067](https://github.com/ray-project/ray/pull/44067), [#44603](https://github.com/ray-project/ray/pull/44603)) - **Evaluation do-over: Deprecate `enable_async_evaluation` option** (in favor of existing `evaluation_paralle _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.11.0]_ ### 2.10.0 — Ray-2.10.0 - Date: 2024-03-21 - Version: 2.10.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.10.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.10.0 - **added** — Ray Data becomes generally available with stability improvements in streaming execution, reading and writing data, better tasks concurrency control, and debuggability improvement with dashboard, logging and metrics visualization - **added** — New API Stack officially announced as alpha for PPO and SAC in RLlib - **added** — Added default autoscaling policy set via num_replicas="auto" in Serve - **added** — Added support for active load shedding via max_queued_requests in Serve - **added** — Added replica queue length caching to the DeploymentHandle scheduler in Serve - **changed** — Renamed max_concurrent_queries to max_ongoing_requests in Serve - **changed** — Renamed target_num_ongoing_requests_per_replica to target_ongoing_requests in Serve - **changed** — Renamed downscale_smoothing_factor to downscaling_factor in Serve - **changed** — Renamed upscale_smoothing_factor to upscaling_factor in Serve - **changed** — max_ongoing_requests (max_concurrent_queries) is now strictly enforced in Serve - **added** — Autoscaler v2 is in alpha for Kuberay with improved observability and stability - **added** — Added support for accelerator types via ScalingConfig(accelerator_type) in Train - **changed** — Revamped XGBoostTrainer and LightGBMTrainer to no longer depend on xgboost_ray and lightgbm_ray - **changed** — Refactored local staging directory to remove the need for local_dir and RAY_AIR_LOCAL_CACHE_DIR in Train/Tune - **added** — Add num_rows_per_file parameter to file-based writes in Ray Data - **added** — Add DataIterator.materialize in Ray Data - **added** — Skip schema call in DataIterator.to_tf if tf.TypeSpec is provided in Ray Data - **added** — Add option to append for Dataset.write_bigquery in Ray Data - **added** — Add backend-specific context manager for train_func for setup/teardown logic in Train - **fixed** — Fix deadlocks caused by Dataset.streaming_split for job hanging #### Release Highlights Ray 2.10 release brings important stability improvements and enhancements to Ray Data, with Ray Data becoming generally available (GA). - [Data] Ray Data becomes generally available with stability improvements in streaming execution, reading and writing data, better tasks concurrency control, and debuggability improvement with dashboard, logging and metrics visualization. - [RLlib] “**New API Stack**” officially announced as alpha for PPO and SAC. - [Serve] Added a default autoscaling policy set via `num_replicas=”auto”` ([#42613](https://github.com/ray-project/ray/issues/42613)). - [Serve] Added support for active load shedding via `max_queued_requests` ([#42950](https://github.com/ray-project/ray/issues/42950)). - [Serve] Added replica queue length caching to the DeploymentHandle scheduler ([#42943](https://github.com/ray-project/ray/pull/42943)). - This should improve overhead in the Serve proxy and handles. - `max_ongoing_requests (max_concurrent_queries)` is also now strictly enforced ([#42947](https://github.com/ray-project/ray/issues/42947)). - If you see any issues, please report them on GitHub and you can disable this behavior by setting: `RAY_SERVE_ENABLE_QUEUE_LENGTH_CACHE=0`. - [Serve] Renamed the following parameters. Each of the old names will be supported for another release before removal. - `max_concurrent_queries` -> `max_ongoing_requests` - `target_num_ongoing_requests_per_replica` -> `target_ongoing_requests` - `downscale_smoothing_factor` -> `downscaling_factor` - `upscale_smoothing_factor` -> `upscaling_factor` - [Core] [Autoscaler v2](https://docs.ray.io/en/master/cluster/kubernetes/user-guides/configuring-autoscaling.html#kuberay-autoscaler-v2) is in alpha and can be tried out with Kuberay. It has improved observability and stability compared to v1. - [Train] Added support for accelerator types via `ScalingConfig(accelerator_type)`. - [Train] Revamped the `XGBoostTrainer` and `LightGBMTrainer` to no longer depend on `xgboost_ray` and `lightgbm_ray`. A new, more flexible API will be released in a future release. - [Train/Tune] Refactored local staging directory to remove the need for `local_dir` and `RAY_AIR_LOCAL_CACHE_DIR`. #### Ray Libraries ##### Ray Data 🎉 New Features: - Streaming execution stability improvement to avoid memory issue, including per-operator resource reservation, streaming generator output buffer management, and better runtime resource estimation (#43026, #43171, #43298, #43299, #42930, #42504) - Metadata read stability improvement to avoid AWS transient error, including retry on application-level exception, spread tasks across multiple nodes, and configure retry interval (#42044, #43216, #42922, #42759). - Allow tasks concurrency control for read, map, and write APIs (#42849, #43113, #43177, #42637) - Data dashboard and statistics improvement with more runtime metrics for each components (#43790, #43628, #43241, #43477, #43110, #43112) - Allow to specify application-level error to retry for actor task (#42492) - Add `num_rows_per_file` parameter to file-based writes (#42694) - Add `DataIterator.materialize` (#43210) - Skip schema call in `DataIterator.to_tf` if `tf.TypeSpec` is provided (#42917) - Add option to append for `Dataset.write_bigquery` (#42584) - Deprecate legacy components and classes (#43575, #43178, #43347, #43349, #43342, #43341, #42936, #43144, #43022, #43023) 💫 Enhancements: - Restructure stdout logging for better readability (#43360) - Add a more performant way to read large TFRecord datasets (#42277) - Modify `ImageDatasource` to use `Image.BILINEAR` as the default image resampling filter (#43484) - Reduce internal stack trace output by default (#43251) - Perform incremental writes to Parquet files (#43563) - Warn on excessive driver memory usage during shuffle ops (#42574) - Distributed reads for _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.10.0]_ ### 2.9.3 — Ray-2.9.3 - Date: 2024-02-22 - Version: 2.9.3 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.9.3 - Permalink: https://whatsnew.fyi/product/ray/releases/2.9.3 - **fixed** — Fix protobuf breaking change by adding a compat layer - **changed** — Bump up task failure logs to warnings to make sure failures could be troubleshooted - **fixed** — Fix placement group leaks - **fixed** — Skip schema call in to_tf if tf.TypeSpec is provided - **fixed** — Skip recording memory spilled stats when get_memory_info_reply is failed - **fixed** — Fix DeploymentStateManager qualifying replicas as running prematurely This patch release contains fixes for Ray Core, Ray Data, and Ray Serve. #### Ray Core 🔨 Fixes: - Fix protobuf breaking change by adding a compat layer. ([#43172](https://github.com/ray-project/ray/pull/43172)) - Bump up task failure logs to warnings to make sure failures could be troubleshooted ([#43147](https://github.com/ray-project/ray/pull/43147)) - Fix placement group leaks ([#42942](https://github.com/ray-project/ray/pull/42942)) #### Ray Data 🔨 Fixes: - Skip `schema` call in `to_tf` if `tf.TypeSpec` is provided ([#42917](https://github.com/ray-project/ray/pull/42917)) - Skip recording memory spilled stats when get_memory_info_reply is failed ([#42824](https://github.com/ray-project/ray/pull/42824)) #### Ray Serve 🔨 Fixes: - Fixing DeploymentStateManager qualifying replicas as running prematurely ([#43075](https://github.com/ray-project/ray/pull/43075)) #### Thanks Many thanks to all those who contributed to this release! @rynewang, @GeneDer, @alexeykudinkin, @edoakes, @c21, @rkooo567 ### 2.9.2 — Ray-2.9.2 - Date: 2024-02-06 - Version: 2.9.2 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.9.2 - Permalink: https://whatsnew.fyi/product/ray/releases/2.9.2 - **fixed** — Fix out of disk test on release branch - **fixed** — Fix failing huggingface test - **fixed** — Fix deadlocks caused by streaming_split - **fixed** — Fix locality config not being respected in DataConfig - **changed** — Improve stability and accuracy for Data+Train benchmark - **fixed** — Add retry for _sample_fragment during ParquetDatasource._estimate_files_encoding_ratio() - **fixed** — Skip recording memory spilled stats when get_memory_info_reply is failed - **fixed** — Pin the fastapi and starlette version to avoid breaking proxy - **fixed** — Fix IS_PYDANTIC_2 logic for pydantic<1.9.0 - **fixed** — Fix missing message body for json log formats This patch release contains fixes for Ray Core, Ray Data, and Ray Serve. #### Ray Core 🔨 Fixes: - Fix out of disk test on release branch (https://github.com/ray-project/ray/pull/42724) #### Ray Data 🔨 Fixes: - Fix failing huggingface test (https://github.com/ray-project/ray/pull/42727) - Fix deadlocks caused by streaming_split (https://github.com/ray-project/ray/pull/42601) (https://github.com/ray-project/ray/pull/42755) - Fix locality config not being respected in DataConfig (https://github.com/ray-project/ray/pull/42204 https://github.com/ray-project/ray/pull/42204) (https://github.com/ray-project/ray/pull/42722) - Stability & accuracy improvements for Data+Train benchmark (https://github.com/ray-project/ray/pull/42027) - Add retry for _sample_fragment during `ParquetDatasource._estimate_files_encoding_ratio()` (https://github.com/ray-project/ray/pull/42759) (https://github.com/ray-project/ray/pull/42774) - Skip recording memory spilled stats when get_memory_info_reply is failed (https://github.com/ray-project/ray/pull/42824) (https://github.com/ray-project/ray/pull/42834) #### Ray Serve 🔨 Fixes: - Pin the fastapi & starlette version to avoid breaking proxy (https://github.com/ray-project/ray/pull/42740 https://github.com/ray-project/ray/pull/42740) - Fix IS_PYDANTIC_2 logic for pydantic<1.9.0 (https://github.com/ray-project/ray/pull/42704) (https://github.com/ray-project/ray/pull/42708) - fix missing message body for json log formats (https://github.com/ray-project/ray/pull/42729) (https://github.com/ray-project/ray/pull/42874) #### Thanks Many thanks to all those who contributed to this release! @c21, @raulchen, @can-anyscale, @edoakes, @peytondmurray, @scottjlee, @aslonnie, @architkulkarni, @GeneDer, @Zandew, @sihanwang41 ### 2.9.1 — Ray-2.9.1 - Date: 2024-01-19 - Version: 2.9.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.9.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.9.1 - **added** — Add debugpy as the ray debugger - **fixed** — Fix task events profile events per task leak - **fixed** — Make sure redis sync context and async context connect to the same redis instance - **fixed** — Retry write if error during file clean up - **fixed** — Improve handling the websocket server disconnect scenario - **fixed** — Fix pydantic config documentation - **changed** — Enable setting queue length response deadline via environment variable - **added** — Add exponential backoff for queue_len_response_deadline_s This patch release contains fixes for Ray Core, Ray Data, and Ray Serve. #### Ray Core 🔨 Fixes: - Adding debupgy as the ray debugger (#42311) - Fix task events profile events per task leak (#42248) - Make sure redis sync context and async context connect to the same redis instance (#42040) #### Ray Data 🔨 Fixes: - [Data] Retry write if error during file clean up (#42326) #### Ray Serve 🔨 Fixes: - Improve handling the websocket server disconnect scenario (#42130) - Fix pydantic config documentation (#42216) - Address issues under high network delays: - Enable setting queue length response deadline via environment variable (#42001) - Add exponential backoff for queue_len_response_deadline_s (#42041) ### 2.9.0 — Ray-2.9.0 - Date: 2023-12-21 - Version: 2.9.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.9.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.9.0 - **security** — Fix security vulnerabilities in Ray Dashboard (CVE-2023-6019, CVE-2023-6020, CVE-2023-6021, CVE-2023-48022, CVE-2023-48023) - **changed** — Upgrade Ray Train support for spot node preemption to handle preemption node failures differently than application errors - **changed** — Make Ray compatible with Pydantic versions <2.0.0 and >=2.5.0 - **added** — Add Ray Dashboard page for Ray Data to monitor real-time execution metrics - **added** — Make streaming generator officially a public API for writing streaming applications on top of Ray via Python generator API - **added** — Add experimental support for Intel GPU accelerators - **added** — Add experimental support for Intel Gaudi Accelerators - **added** — Add experimental support for Huawei Ascend NPU accelerators - **added** — Add concurrency argument to Ray Data map-like APIs to replace ComputeStrategy - **added** — Allow task failures during Ray Data execution - **changed** — Support PyArrow 14.0.1 in Ray Data - **added** — Add new API for reading and writing Datasource in Ray Data - **added** — Enable group-by over multiple keys in Ray Data datasets - **added** — Add support for multiple group keys in map_groups for Ray Data - **added** — Introduce logging config in Ray Serve to set different logging parameters for different applications and deployments - **added** — Add gRPC context object into gRPC deployments in Ray Serve for users to set custom code and details back to the client - **added** — Introduce experimental runtime environment feature in Ray Serve to run applications in different containers with different images - **added** — Support reading Result from cloud storage in Ray Train and Ray Tune - **removed** — Remove Legacy Trainers from Ray Train - **removed** — Delete legacy TuneClient and TuneServer APIs from Ray Tune - **removed** — Delete legacy Searchers from Ray Tune #### Release Highlights - This release contains fixes for the Ray Dashboard. Additional context can be found here:   - Ray Train has now upgraded support for spot node preemption -- allowing Ray Train to handle preemption node failures differently than application errors. - Ray is now compatible with Pydantic versions <2.0.0 and >=2.5.0, addressing a piece of user feedback we’ve consistently received. - The Ray Dashboard now has a page for Ray Data to monitor real-time execution metrics. - [Streaming generator](https://docs.ray.io/en/latest/ray-core/ray-generator.html) is now officially a public API (#41436, #38784). Streaming generator allows writing streaming applications easily on top of Ray via Python generator API and has been used for Ray Serve and Ray data for several releases. See the [documentation](https://docs.ray.io/en/master/ray-core/ray-generator.html) for details.  - We’ve added experimental support for new accelerators: Intel GPU (#38553), Intel Gaudi Accelerators (#40561), and Huawei Ascend NPU (#41256). #### Ray Libraries ##### Ray Data 🎉 New Features: * Add the dashboard for Ray Data to monitor real-time execution metrics and log file for debugging (). * Introduce `concurrency` argument to replace `ComputeStrategy` in map-like APIs (#41461) * Allow task failures during execution (#41226) * Support PyArrow 14.0.1 (#41036) * Add new API for reading and writing Datasource () * Enable group-by over multiple keys in datasets (#37832) * Add support for multiple group keys in `map_groups` (#40778) 💫 Enhancements: - Optimize `OpState.outqueue_num_blocks` (#41748) - Improve stall detection for `StreamingOutputsBackpressurePolicy` (#41637) - Enable read-only Datasets to be executed on new execution backend (#41466, #41597) - Inherit block size from downstream ops (#41019) - Use runtime object memory for scheduling (#41383) - Add retries to file writes (#41263) - Make range datasource streaming (#41302) - Test core performance metrics (#40757) - Allow `ConcurrencyCapBackpressurePolicy._cap_multiplier` to be set to 1.0 (#41222) - Create `StatsManager` to manage `_StatsActor` remote calls (#40913) - Expose `max_retry_cnt` parameter for `BigQuery` Write (#41163) - Add rows outputted to data metrics (#40280) - Add fault tolerance to remote tasks (#41084) - Add operator-level dropdown to ray data overview (#40981) - Avoid slicing too-small blocks (#40840) - Ray Data jobs detail table (#40756) - Update default shuffle block size to 1GB (#40839) - Log progress bar to data logs (#40814) - Operator level metrics (#40805) 🔨 Fixes: - Partial fix for `Dataset.context` not being sealed after creation (#41569) - Fix the issue that `DataContext` is not propagated when using `streaming_split` (#41473) - Fix Parquet partition filter bug (#40947) - Fix split read output blocks (#41070) - Fix `BigQueryDatasource `fault tolerance bugs (#40986) 📖 Documentation: - Add example of how to read and write custom file types (#41785) - Fix `ray.data.read_databricks_tables` doc (#41366) - Add `read_json` docs example for setting PyArrow block size when reading large files (#40533) - Add `AllToAllAPI` to dataset methods (#40842) ##### Ray Train 🎉 New Features: - Support reading `Result` from cloud storage (#40622) 💫 Enhancements: - Sort local Train workers by GPU ID (#40953) - Improve logging for Train worker scheduling information (#40536) - Load the latest unflattened metrics with `Result.from_path` (#40684) - Skip incrementing failure counter on preemption node died failures (#41285) - Update TensorFlow `ReportCheckpointCallback` to _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.9.0]_ ### 2.8.1 — Ray-2.8.1 - Date: 2023-12-01 - Version: 2.8.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.8.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.8.1 - **fixed** — Prevent state API from reading files outside the Ray log directory - **changed** — Migrate Logs page to use state api #### Release Highlights The Ray 2.8.1 patch release contains fixes for the Ray Dashboard. Additional context can be found here: https://www.anyscale.com/blog/update-on-ray-cves-cve-2023-6019-cve-2023-6020-cve-2023-6021-cve-2023-48022-cve-2023-48023 #### Ray Dashboard 🔨 Fixes: [core][state][log] Cherry pick changes to prevent state API from reading files outside the Ray log directory (#41520) [Dashboard] Migrate Logs page to use state api. (#41474) (#41522) ### 2.8.0 — Ray-2.8.0 - Date: 2023-11-03 - Version: 2.8.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.8.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.8.0 - **added** — Add support for shuffling input files in Ray Data - **added** — Support streaming read of PyTorch dataset - **added** — Add BigQuery datasource for Ray Data - **added** — Add Databricks table and SQL datasource for Ray Data - **added** — Add inverse transform functionality to LabelEncoder - **added** — Add function arg params to Dataset.map and Dataset.flat_map - **added** — Add initial support for scheduling workers on neuron_cores in Ray Train - **added** — Add new Java APIs that aligns with the Ray Serve 2.x APIs - **added** — Add new metrics for Ray Data in the Metrics tab on the Ray Dashboard - **added** — Ray Core now supports profiling GPU tasks or actors using Nvidia Nsight - **changed** — Update PyTorch Lightning import path to support both pytorch_lightning and lightning - **changed** — Propagate driver DataContext to RayTrainWorkers - **changed** — The Serve REST API is now accessible through the dashboard port which defaults to 8265 - **changed** — Hard deprecate DatasetPipeline - **changed** — Remove BulkExecutor code path - **changed** — Remove legacy iteration code path in Ray Data - **fixed** — Fix child process leak issue from Ray worker that leaks GPU memory - **fixed** — Fix job page excessive loading time issue when Ray HA cluster restarts a head node - **fixed** — Fix bug where _StatsActor errors with PandasBlock - **deprecated** — Python 3.7 support is officially deprecated from Ray - **deprecated** — Deprecate previously experimental DAG API for deployment graphs in Ray Serve - **deprecated** — Deprecate single-application config file format in Ray Serve - **deprecated** — Deprecate accessing the Serve REST API through the dashboard agent port - **deprecated** — Deprecate LightningTrainer, AccelerateTrainer, TransformersTrainer - **deprecated** — Deprecate legacy DatasetConfig in Ray Train - **removed** — Remove deprecated Ray Serve 1.x APIs including Deployment.deploy, Deployment.delete, and Deployment.get_handle - **removed** — Move 24 algorithms into rllib_contrib #### Release Highlights This release features stability improvements and API clean-ups across the Ray libraries. - In Ray Serve, we are deprecating the previously experimental DAG API for deployment graphs. Model composition will be supported through [deployment handles](https://docs.ray.io/en/latest/serve/model_composition.html) providing more flexibility and stability. The previously deprecated Ray Serve 1.x APIs have also been removed. We’ve also added a new Java APIs that aligns with the Ray Serve 2.x APIs. More API changes in the release notes below. - In RLlib, we’ve moved 24 algorithms into `rllib_contrib` (still available within RLlib for Ray 2.8). - We’ve added support for PyTorch-compatible input files shuffling for Ray Data. This allows users to randomly shuffle input files for better model training accuracy. This release also features new Ray Data datasources for Databricks and BigQuery. - On the Ray Dashboard, we’ve added new metrics for Ray Data in the Metrics tab. This allows users to monitor Ray Data workload including real time metrics of cluster memory, CPU, GPU, output data size, etc. See [the doc](https://docs.ray.io/en/master/data/performance-tips.html#monitoring-your-application) for more details. - Ray Core now supports profiling GPU tasks or actors using Nvidia Nsight. See [the documentation](https://docs.ray.io/en/master/ray-observability/user-guides/profiling.html?highlight=nsight#nsight-system-profiler) for instructions. - We fixed 2 critical bugs raised by many kuberay / ML library users, including a child process leak issue from Ray worker that leaks the GPU memory (#40182) and an job page excessive loading time issue when Ray HA cluster restarts a head node (#40742) - Python 3.7 support is officially deprecated from Ray. #### Ray Libraries ##### Ray Data 🎉 New Features: - Add support for shuffling input files (#40154) - Support streaming read of PyTorch dataset (#39554) - Add BigQuery datasource (#37380) - Add Databricks table / SQL datasource (#39852) - Add inverse transform functionality to LabelEncoder (#37785) - Add function arg params to `Dataset.map` and `Dataset.flat_map` (#40010) 💫Enhancements: - Hard deprecate `DatasetPipeline` (#40129) - Remove `BulkExecutor` code path (#40200) - Deprecate extraneous `Dataset` parameters and methods (#40385) - Remove legacy iteration code path (#40013) - Implement streaming output backpressure (#40387) - Cap op concurrency with exponential ramp-up (#40275) - Store ray dashboard metrics in `_StatsActor` (#40118) - Slice output blocks to respect target block size (#40248) - Drop columns before grouping by in `Dataset.unique()` (#40016) - Standardize physical operator runtime metrics (#40173) - Estimate blocks for limit and union operator (#40072) - Store bytes spilled/restored after plan execution (#39361) - Optimize `sample_boundaries` in `SortTaskSpec` (#39581) - Optimization to reduce ArrowBlock building time for blocks of size 1 (#38833) 🔨 Fixes: - Fix bug where `_StatsActor` errors with `PandasBlock` (#40481) - Remove deprecated `do_write` (#40422) - Improve error message when reading HTTP files (#40462) - Add flag to skip `get_object_locations` for metrics (#39884) - Fall back to fetch files info in parallel for multiple directories (#39592) - Replace deprecated `.pieces` with updated `.fragments` (#39523) - Backwards compatibility for `Preprocessor` that have been fit in older versions (#39173) - Removing unnecessary data copy in `convert_udf_returns_to_numpy` (#39188) - Do not eagerly free root `RefBundles` (#39016) 📖Documentation: - Remove out-of-date Data examples (#40127) - Remove unused and outdated source examples (#40271) ##### Ray Train 🎉 New Features: - Add initial support for scheduling workers on neuron_cores (#39091) 💫Enhancements: - Update PyTorch Lightning import path to support both `pytorch_lightning` and `lightning` (#39841, #40266) - Propagate driver `DataConte _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.8.0]_ ### 2.7.1 — Ray-2.7.1 - Date: 2023-10-09 - Version: 2.7.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.7.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.7.1 - **added** — Add an `application` tag to the `ray_serve_num_http_requests` metric - **fixed** — Fix a bug where no data shows up on the `Error QPS per Application` panel in the Ray Dashboard - **fixed** — DreamerV3: Enable support for continuous actions - **fixed** — Fix a bug where setting a local storage path on Windows errors - **fixed** — Fix a broken `Trial.node_ip` property - **fixed** — Fix a segfault when a streaming generator and actor cancel is used together - **fixed** — Fix autoscaler SDK accidentally initializing a ray worker leading to leaked driver showing up in the dashboard - **added** — Add a new user guide for the vSphere cluster launcher - **fixed** — Fix bugs for the vSphere cluster launcher - **fixed** — Fix a bug where `ray start` would occasionally fail with `ValueError: `acceleratorType` should match v(generation)-(cores/chips).` - **changed** — Improve cluster page UI in Dashboard - **fixed** — Fix a bug where overview page UI will crash in Dashboard - **fixed** — Fix Ray cluster stability on a high latency environment #### Release Highlights * Ray Serve: * Added an `application` tag to the `ray_serve_num_http_error_requests` metric * Fixed a bug where no data shows up on the `Error QPS per Application` panel in the Ray Dashboard * RLlib: * DreamerV3: Bug fix enabling support for continuous actions. * Ray Train: * Fix a bug where setting a local storage path on Windows errors ([#39951](https://github.com/ray-project/ray/pull/39951)) * Ray Tune: * Fix a broken `Trial.node_ip` property ([#40028](https://github.com/ray-project/ray/pull/40028)) * Ray Core: * Fixes a segfault when a streaming generator and actor cancel is used together * Fix autoscaler sdk accidentally initialize ray worker leading to leaked driver showing up in the dashboard. * Added a new user guide and fixes for the vSphere cluster launcher. * Fixed a bug where `ray start `would occasionally fail with `ValueError: `acceleratorType` should match v(generation)-(cores/chips).` * Dashboard: * Improvement on cluster page UI * Fix a bug that overview page UI will crash #### Ray Libraries ##### Ray Serve 🔨 Fixes: * Fixed a bug where no data shows up on the `Error QPS per Application` panel in the Ray Dashboard ##### RLlib 🔨 Fixes: * DreamerV3: Bug fix enabling support for continuous actions ([39751](https://github.com/ray-project/ray/issues/39751)). #### Ray Core and Ray Clusters 🔨 Fixes: * Fixed Ray cluster stability on a high latency environment Thanks Many thanks to all those who contributed to this release! @chaowanggg, @allenwang28, @shrekris-anyscale, @GeneDer, @justinvyu, @can-anyscale, @edoakes, @architkulkarni, @rkooo567, @rynewang, @rickyyx, @sven1977 ### 2.7.0 — Ray-2.7.0 - Date: 2023-09-17 - Version: 2.7.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.7.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.7.0 - **changed** — Rebrand Ray AI Runtime (AIR) to Ray AI Libraries and consolidate the ray.air namespace into ray.data, ray.train, and ray.tune - **added** — Ray Serve and Ray Train PyTorch support are now generally available - **added** — Introduce new backwards-compatible DeploymentHandle API in Ray Serve to unify various existing Handle APIs - **added** — Add high performant gRPC proxy to Ray Serve to serve gRPC requests - **added** — Consolidate various PyTorch-based trainers into TorchTrainer in Ray Train - **added** — Introduce new train.Checkpoint API providing consolidated way of interacting with remote and local storage - **added** — Add initial integrations with TPUs and AWS accelerators in Ray Core to natively detect these devices and schedule tasks/actors onto them - **added** — Add official support for actor task cancellation in Ray Core - **added** — Add experimental streaming generator in Ray Core that supports streaming response to the caller - **added** — Integrate Ray Core streaming generator API by default in Ray Data to reduce memory footprint throughout the data pipeline - **added** — Add Dataset.write_images to write images - **added** — Add Dataset.write_sql() to write SQL databases - **added** — Support sort on multiple keys in Ray Data - **added** — Support reading and writing JSONL file format in Ray Data - **added** — Support class constructor args for Dataset.map() and flat_map() - **added** — Implement streamed read from Hugging Face Dataset - **changed** — Read data with multi-threading for FileBasedDataSource in Ray Data - **changed** — Add partition_filter parameter to read_parquet - **changed** — Move away from LightningTrainer in favor of TorchTrainer as the recommended way of running distributed PyTorch Lightning - **deprecated** — Ray Train Preprocessors and Predictors are deprecated in favor of Ray Data #### Release Highlights Ray 2.7 release brings important stability improvements and enhancements to Ray libraries, with Ray Train and Ray Serve becoming generally available. Ray 2.7 is accompanied with a GA release of KubeRay. * Following user feedback, we are rebranding “Ray AI Runtime (AIR)” to “Ray AI Libraries”. Without reducing any of the underlying functionality of the original Ray AI runtime vision as put forth in Ray 2.0, the underlying namespace (ray.air) is consolidated into ray.data, ray.train, and ray.tune. This change reduces the friction for new machine learning (ML) practitioners to quickly understand and leverage Ray for their production machine learning use cases. * With this release, Ray Serve and Ray Train’s Pytorch support are becoming Generally Available -- indicating that the core APIs have been marked stable and that both libraries have undergone significant production hardening. * In Ray Serve, we are introducing a new backwards-compatible `DeploymentHandle` API to unify various existing Handle APIs, a high performant gRPC proxy to serve gRPC requests through Ray Serve, along with various stability and usability improvements. * In Ray Train, we are consolidating various Pytorch-based trainers into the TorchTrainer, reducing the amount of refactoring work new users needed to scale existing training scripts. We are also introducing a new train.Checkpoint API, which provides a consolidated way of interacting with remote and local storage, along with various stability and usability improvements. * In Ray Core, we’ve added initial integrations with TPUs and AWS accelerators, enabling Ray to natively detect these devices and schedule tasks/actors onto them. Ray Core also officially now supports actor task cancellation and has an experimental streaming generator that supports streaming response to the caller. Take a look at our [refreshed documentation](https://docs.ray.io/en/releases-2.7.0) and the [Ray 2.7 migration guide](https://docs.google.com/document/d/1J-09US8cXc-tpl2A1BpOrlHLTEDMdIJp6Ah1ifBUw7Y/view#heading=h.3eeweptnwn6p) and let us know your feedback! #### Ray Libraries ##### Ray AIR 🏗 Architecture refactoring: * **Ray AIR namespace**: We are sunsetting the "Ray AIR" concept and namespace (#39516, #38632, #38338, #38379, #37123, #36706, #37457, #36912, #37742, #37792, #37023). The changes follow the proposal outlined in [this REP](https://github.com/ray-project/enhancements/pull/36). * **Ray Train Preprocessors, Predictors**: We now recommend using Ray Data instead of Preprocessors (#38348, #38518, #38640, #38866) and Predictors (#38209). ##### Ray Data 🎉 New Features: * In this release, we’ve integrated the Ray Core streaming generator API by default, which allows us to reduce memory footprint throughout the data pipeline (#37736). * Avoid unnecessary data buffering between `Read` and `Map` operator (zero-copy fusion) (#38789) * Add `Dataset.write_images` to write images (#38228) * Add `Dataset.write_sql()` to write SQL databases (#38544) * Support sort on multiple keys (#37124) * Support reading and writing JSONL file format (#37637) * Support class constructor args for `Dataset.map()` and `flat_map()` (#38606) * Implement streamed read from Hugging Face Dataset (#38432) 💫Enhancements: * Read data with multi-threading for `FileBasedDataSource` (#39493) * Optimization to reduce `ArrowBlock` building time for blocks of size 1 (#38988) * Add `partition_filter` parameter to `read_parquet `(#38479) * Apply limit to `Dataset.take()` and related methods (#38677) * Postpone `reader.get_read_tasks` until execution (#38373) * Lazily construct metadata providers (#38198) * Support writing each block to a separate file (#37986) * Make `iter_batches` an Iterable (#37881) * Remove default limit on `Dataset.to_pandas()` (#37420) * Add `Dataset.to_dask()` parameter to toggle consistent metadata check (#37163) * Add `Datasource. _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.7.0]_ ### 2.6.3 — Ray-2.6.3 - Date: 2023-08-15 - Version: 2.6.3 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.6.3 - Permalink: https://whatsnew.fyi/product/ray/releases/2.6.3 - **fixed** — Fix memory leak from the end of object stream object in streaming generators - **fixed** — Fix serve run help message - **fixed** — Decrement ray_serve_deployment_queued_queries when client disconnects - **added** — Add Learner API documentation for RLlib The Ray 2.6.3 patch release contains fixes for Ray Serve, and Ray Core streaming generators. ##### Ray Core 🔨 Fixes: * [Core][Streaming Generator] Fix memory leak from the end of object stream object #38152 (#38206) ##### Ray Serve 🔨 Fixes: * [Serve] Fix `serve run` help message (#37859) (#38018) * [Serve] Decrement `ray_serve_deployment_queued_queries` when client disconnects (#37965) (#38020) ##### RLib 📖 Documentation: * [RLlib][docs] Learner API Docs (#37729) (#38137) ### 2.6.2 — Ray-2.6.2 - Date: 2023-08-03 - Version: 2.6.2 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.6.2 - Permalink: https://whatsnew.fyi/product/ray/releases/2.6.2 - **fixed** — Pass logs through if sphinx-doctest is running - **fixed** — Pick GCP cluster launcher tests and fix - **fixed** — Apply request_timeout_s from Serve config to the cluster - **fixed** — Fix pyarrow lazy import The Ray 2.6.2 patch release contains a critical fix for ray's logging setup, as well fixes for Ray Serve, Ray Data, and Ray Job. ##### Ray Core 🔨 Fixes: * [Core] Pass logs through if sphinx-doctest is running (#36306) (#37879) * [cluster-launcher] Pick GCP cluster launcher tests and fix (#37797) ##### Ray Serve 🔨 Fixes: * [Serve] Apply `request_timeout_s` from Serve config to the cluster (#37884) (#37903) ##### Ray Air 🔨 Fixes: * [air] fix pyarrow lazy import (#37670) (#37883) ### 2.6.1 — Ray-2.6.1 - Date: 2023-07-24 - Version: 2.6.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.6.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.6.1 - **fixed** — Fix cluster launcher failing to start multi-node clusters in Ray 2.6.0 - **fixed** — Fix autoscaler env variable overwrite not being usable if the command itself uses the env variable - **fixed** — Fix Serve enum to_proto compatibility with Python 3.11 The Ray 2.6.1 patch release contains a critical fix for cluster launcher, and compatibility update for Ray Serve protobuf definition with python 3.11, as well doc improvements. ⚠️ Cluster launcher in Ray 2.6.0 fails to start multi-node clusters. Please update to 2.6.1 if you plan to use 2.6.0 cluster launcher. ##### Ray Core 🔨 Fixes: * [core][autoscaler] Fix env variable overwrite not able to be used if the command itself uses the env #37675 ##### Ray Serve 🔨 Fixes: * [serve] Cherry-pick Serve enum to_proto fixes for Python 3.11 #37660 ##### Ray Air 📖Documentation: * [air][doc] Update docs to reflect head node syncing deprecation #37475 ### 2.6.0 — Ray-2.6.0 - Date: 2023-07-21 - Version: 2.6.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.6.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.6.0 - **added** — Support for HTTP streaming response and WebSockets is now enabled by default in Ray Serve - **added** — Ray Serve @serve.batch-decorated methods can now stream responses - **added** — Support for restoring Results from local trial directories - **added** — New streaming integration of Ray Data and Ray Train allowing streaming data ingestion for model training and per-epoch data preprocessing - **added** — Add Dataset.unique() method - **added** — Add option for parallelizing post-collation data batch operations in DataIterator.iter_batches() - **added** — LightningTrainer support for DeepSpeedStrategy - **changed** — Users are now required to provide cloud storage or NFS path for distributed training or tuning jobs instead of a local path - **changed** — Enable execution optimizer by default in Ray Data - **changed** — Enforce strict mode batch format for DataIterator.iter_batches() - **changed** — Spread map task stages by default for arg size <50MB - **changed** — Unify Lightning and AIR CheckpointConfig - **changed** — Add support for custom pipeline class in TransformersPredictor - **deprecated** — DatasetPipeline API is deprecated in favor of Dataset with streaming execution - **deprecated** — BatchPredictor is deprecated - **fixed** — Fix infinite recursion in log redirection - **fixed** — Remove temporary checkpoint directories after restore - **fixed** — Fix bug in execution for actor re-use - **removed** — Remove ray.data.range_arrow() - **removed** — Remove deprecated mlflow and wandb integrations #### Release Highlights * **Serve**: Better streaming support -- In this release, Support for HTTP streaming response and WebSockets is now on by default. Also, `@serve.batch`-decorated methods can stream responses. * **Train and Tune**: Users are now expected to provide cloud storage or NFS path for distributed training or tuning jobs instead of a local path. This means that results written to different worker machines will not be directly synced to the head node. Instead, this will raise an error telling you to switch to one of the recommended alternatives: cloud storage or NFS. Please see https://github.com/ray-project/ray/issues/37177 if you have questions. * **Data**: We are introducing a new streaming integration of Ray Data and Ray Train. This allows streaming data ingestion for model training, and enables per-epoch data preprocessing. The DatasetPipeline API is also being deprecated in favor of Dataset with streaming execution. * **RLlib**: Public alpha release for the new multi-gpu Learner API that is less complex and more powerful compared to our previous solution ([blogpost](https://www.anyscale.com/blog/introducing-rllib-multi-gpu-stack-for-cost-efficient-scalable-multi-gpu-rl)). This is used under PPO algorithm by default. #### Ray Libraries ##### Ray AIR 🎉 **New Features**: * Added support for restoring Results from local trial directories. (#35406) 💫 **Enhancements**: * [Train/Tune] Disable Train/Tune syncing to head node (#37142) * [Train/Tune] Introduce new console output progress reporter for Train and Tune (#35389, #36154, #36072, #35770, #36764, #36765, #36156, #35977) * [Train/Data] New Train<>Data streaming integration (#35236, #37215, #37383) 🔨 **Fixes**: * Pass on KMS-related kwargs for s3fs (#35938) * Fix infinite recursion in log redirection (#36644) * Remove temporary checkpoint directories after restore (#37173) * Removed actors that haven't been started shouldn't be tracked (#36020) * Fix bug in execution for actor re-use (#36951) * Cancel `pg.ready()` task for pending trials that end up reusing an actor (#35748) * Add case for `Dict[str, np.array]` batches in `DummyTrainer` read bytes calculation (#36484) 📖 **Documentation**: * Remove experimental features page, add github issue instead (#36950) * Fix batch format in `dreambooth` example (#37102) * Fix Checkpoint.from_checkpoint docstring (#35793) 🏗 **Architecture refactoring**: * Remove deprecated mlflow and wandb integrations (#36860, #36899) * Move constants from tune/results.py to air/constants.py (#35404) * Clean up a few checkpoint related things. (#35321) ##### Ray Data 🎉 **New Features**: * New streaming integration of Ray Data and Ray Train. This allows streaming data ingestion for model training, and enables per-epoch data preprocessing. (#35236) * Enable execution optimizer by default (#36294, #35648, #35621, #35952) * Deprecate DatasetPipeline (#35753) * Add `Dataset.unique()` (#36655, #36802) * Add option for parallelizing post-collation data batch operations in `DataIterator.iter_batches()` (#36842) (#37260) * Enforce strict mode batch format for `DataIterator.iter_batches()` (#36686) * Remove `ray.data.range_arrow()` (#35756) 💫 **Enhancements**: * Optimize block prefetching (#35568) * Enable isort for data directory (#35836) * Skip writing a file for an empty block in `Dataset.write_datasource()` (#36134) * Remove shutdown logging from StreamingExecutor (#36408) * Spread map task stages by default for arg size <50MB (#36290) * Read->SplitBlocks to ensure requested read parallelism is always met (#36352) * Support partial execution in `Dataset.schema()` with new execution plan optimizer (#36740) * Propagate iter stats for `Dataset.streaming_split()` (#36908) * Cache the computed schema to avoid re-executing (#37103) 🔨 **Fixes**: * Support sub-progress bars on AllToAllOperators with optimizer enabled (#34997) * Fix DataContext not propagated properly _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.6.0]_ ### 2.5.1 — Ray-2.5.1 - Date: 2023-06-21 - Version: 2.5.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.5.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.5.1 - **added** — Build Python wheels on Mac OS for Python 3.11 - **fixed** — Don't error on eventual success when running with auto-recovery in Ray Train - **fixed** — Fix a bug that can cause undefined behavior when clusters attempt to scale up aggressively in Autoscaler - **fixed** — Fix mypy error where module "ray" does not explicitly export attribute "remote" The Ray 2.5.1 patch release adds wheels for MacOS for Python 3.11. It also contains fixes for multiple components, along with fixes for our documentation. ##### Ray Train 🔨 Fixes: * Don't error on eventual success when running with auto-recovery (#36266) ##### Ray Core 🎉 New Features: * Build Python wheels on Mac OS for Python 3.11 (#36373) 🔨 Fixes: * [Autoscaler] Fix a bug that can cause undefined behavior when clusters attempt to scale up aggressively. (#36241) * Fix mypy error: Module "ray" does not explicitly export attribute "remote" (#36356) ### 2.5.0 — Ray-2.5.0 - Date: 2023-06-08 - Version: 2.5.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.5.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.5.0 - **added** — Support for checkpointing distributed models in Ray Train - **added** — PyTorch Lightning FSDP support in LightningTrainer to enable training large models on Ray Train - **added** — Support for streaming responses and model multiplexing in Ray Serve and Core for LLM applications - **changed** — Enable strict mode by default in Ray Data, requiring schemas for all Datasets and removing support for standalone Python objects - **changed** — Set default batch format to NumPy in Ray Data for better performance in batch inference - **added** — Support for multi-GPU training in RLlib - **added** — Enable lightweight resource broadcasting feature in Ray Core to improve reliability and scalability - **added** — Add column API to Dataset - **added** — Configure progress bars via DataContext - **added** — Support using concurrent actors for ActorPool - **added** — Add take_batch API for collecting data in the same format as iter_batches and map_batches - **added** — Add alias parameters to the aggregate function and add quantile function - **changed** — Add RunConfig.storage_path to replace SyncConfig.upload_dir and RunConfig.local_dir - **changed** — Use Ray storage URI as default storage path if configured - **changed** — Use wait based prefetcher by default in Ray Data - **changed** — Require compute spec to be explicitly spelled out - **fixed** — Fix bugs in handling of nested ndarrays and other complex object types - **fixed** — Fix backpressure handling when reading directly from input datasource - **fixed** — Fix backpressure handling of queued actor pool tasks - **fixed** — Fix row count after applying filter The Ray 2.5 release features focus on a number of enhancements and improvements across the Ray ecosystem, including: * Training LLMs with Ray Train: New support for checkpointing distributed models, and Pytorch Lightning FSDP to enable training large models on Ray Train’s LightningTrainer * LLM applications with Ray Serve & Core: New support for streaming responses and model multiplexing * Improvements to Ray Data: In 2.5, [strict mode ](https://docs.ray.io/en/master/data/faq.html#migrating-to-strict-mode)is enabled by default. This means that schemas are required for all Datasets, and standalone Python objects are no longer supported. Also, the default batch format is fixed to NumPy, giving better performance for [batch inference](https://docs.ray.io/en/master/data/batch_inference.html). * RLlib enhancements: New support for multi-gpu training, along with [ray-project/rllib-contrib](https://github.com/ray-project/rllib-contrib) to contain the community contributed algorithms * Core enhancements: Enable new feature of lightweight resource broadcasting to improve reliability and scalability. Add many enhancements for Core reliability, logging, scheduler, and worker process. #### Ray Libraries ##### Ray AIR 💫Enhancements: * Experiment restore stress tests (#33706) * Context-aware output engine * Add parameter columns to status table (#35388) * Context-aware output engine: Add docs, experimental feature docs, prepare default on (#35129) * Fix trial status at end (more info + cut off) (#35128) * Improve leaked mentions of Tune concepts (#35003) * Improve passed time display (#34951) * Use flat metrics in results report, use Trainable._progress_metrics (#35035) * Print experiment information at experiment start (#34952) * Print single trial config + results as table (#34788) * Print out worker ip for distributed train workers. (#33807) * Minor fix to print configuration on start. (#34575) * Check `air_verbosity` against None. (#33871) * Better wording for empty config. (#33811) * Flatten config and metrics before passing to mlflow (#35074) * Remote_storage: Prefer fsspec filesystems over native pyarrow (#34663) * Use filesystem wrapper to exclude files from upload (#34102) * GCE test variants for air_benchmark and air_examples (#34466) * New storage path configuration * Add `RunConfig.storage_path` to replace `SyncConfig.upload_dir` and `RunConfig.local_dir`. (#33463) * Use Ray storage URI as default storage path, if configured [no_early_kickoff] (#34470) * Move to new storage_path API in tests and examples (#34263) 🔨 Fixes: * Store unflattened metrics in _TrackedCheckpoint (#35658) (#35706) * Fix `test_tune_torch_get_device_gpu` race condition (#35004) * Deflake test_e2e_train_flow.py (#34308) * Pin deepspeed version for now to unblock ci. (#34406) * Fix AIR benchmark configuration link failure. (#34597) * Fix unused config building function in lightning MNIST example. 📖Documentation: * Change doc occurrences of ray.data.Dataset to ray.data.Datastream (#34520) * DreamBooth example: Fix code for batch size > 1 (#34398) * Synced tabs in AIR getting started (#35170) * New Ray AIR link for try it out (#34924) * Correctly Render the Enumerate Numbers in `convert_torch_code_to_ray_air` (#35224) ##### Ray Data Processing 🎉 New Features: * Implement Strict Mode and enable it by default. * Add column API to Dataset (#35241) * Configure progress bars via DataContext (#34638) * Support using concurrent actors for ActorPool (#34253) * Add take_batch API for collecting data in the same format as iter_batches and map_batches (#34217) 💫Enhancements: * Improve map batches error message for strict mode migration (#35368) * Improve docstring and warning message for from_huggingface (#35206) * Improve notebook widget display (#34359) * Implement some operator fusion logic for the new backend (#35178 #34847) * Use wait based prefetcher by d _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.5.0]_ ### 2.4.0 — Ray-2.4.0 - Date: 2023-04-25 - Version: 2.4.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.4.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.4.0 - **added** — Introduce LightningTrainer to scale PyTorch Lightning on Ray - **added** — Release AccelerateTrainer to run HuggingFace Accelerate and DeepSpeed on Ray - **added** — Add examples for GPT-J fine-tuning with Microsoft DeepSpeed and Ray Train - **added** — Add examples for GPT-J-6B batch prediction with Ray Data - **added** — Add examples for GPT-J-6B serving with Ray Serve - **added** — Add examples for Stable Diffusion Dreambooth fine-tuning with Ray Train - **added** — Add examples for Stable Diffusion batch prediction with Ray Data - **added** — Add examples for Stable Diffusion serving with Ray Serve - **added** — Enable streaming execution by default for more efficient data processing pipelines - **added** — Implement asynchronous batch prefetching in Dataset.iter_batches - **added** — Add support for reading SQL databases with Ray Data - **added** — Add support for reading WebDataset format - **added** — Add multi-app CLI and REST API support to Ray Serve - **added** — Add TorchDetectionPredictor - **added** — Add collate_fn parameter to iter_torch_batches - **changed** — Make Preprocessor.transform lazy by default - **changed** — Make BatchPredictor lazy - **changed** — Enhance logging and metrics for Serve applications - **changed** — Replace ray_lightning integration with LightningTrainer for latest PyTorch Lightning compatibility - **deprecated** — Ray 2.4 is the last version that supports Python 3.6 #### Ray 2.4 - Generative AI and LLM support Over the last few months, we have seen a flurry of innovative activity around [generative AI models](https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-generative-ai) and [large language models (LLM)](https://en.wikipedia.org/wiki/Large_language_model). To continue our effort to ensure Ray provides a pivotal compute substrate for [generative AI workloads](https://www.anyscale.com/blog/ray-common-production-challenges-for-generative-ai-infrastructure) and addresses the challenges (as explained in our [blog series](https://www.anyscale.com/blog/ray-common-production-challenges-for-generative-ai-infrastructure)), we have invested engineering efforts in this release to ensure that these open source LLM models and workloads are accessible to the open source community and performant with Ray. This release includes new examples for training, batch inference, and serving with your own LLM. ##### Generative AI and LLM Examples * [GPT-J (LLM) fine-tuning with Microsoft DeepSpeed and Ray Train](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/gptj_deepspeed_fine_tuning.html) * [GPT-J-6B Batch Prediction with Ray Data](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/gptj_batch_prediction.html) * [GPT-J-6B Serving with Ray Serve](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/gptj_serving.html) * [Stable Diffusion (Dreambooth) fine-tuning with Ray Train](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/dreambooth_finetuning.html) * [Stable Diffusion Batch Prediction with Ray Data ](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/stablediffusion_batch_prediction.html) * [Stable Diffusion Serving with Ray Serve](https://docs.ray.io/en/releases-2.4.0/serve/tutorials/stable-diffusion.html) ##### Ray Train enhancements * We're introducing the [LightningTrainer](https://docs.ray.io/en/releases-2.4.0/train/api/doc/ray.train.lightning.LightningTrainer.html), allowing you to scale your [PyTorch Lightning](https://lightning.ai/docs/pytorch/stable//index.html) on Ray. As part of our continued effort for seamless integration and ease of use, we have enhanced and replaced our existing ray_lightning integration, which was widely adopted, with the latest changes to Pytorch Lighting. * we’re releasing an [AccelerateTrainer](https://docs.ray.io/en/releases-2.4.0/train/api/doc/ray.train.huggingface.accelerate.AccelerateTrainer.html), allowing you to run [HuggingFace Accelerate](https://huggingface.co/docs/accelerate) and [DeepSpeed](https://huggingface.co/docs/accelerate/usage_guides/deepspeed) on Ray with minimal code changes. This Trainer integrates with the rest of the Ray ecosystem—including the ability to run distributed [hyperparameter tuning](https://docs.ray.io/en/latest/tune/index.html) with each trial being a distributed training job. ##### Ray Data highlights * Streaming execution is enabled by default, providing users with a more efficient data processing pipeline that can handle larger datasets and minimize memory consumption. Check out the docs here: ([doc](https://docs.ray.io/en/releases-2.4.0/data/dataset-internals.html#streaming-execution)) * Note that this means data output may no longer preserve the original order in more cases. To retain the original ordering properties of Ray Data prior to 2.4, you can set the config `ray.data.DatasetContext.get_current().execution_options.preserve_order = True`. * We've implemented asynchronous batch prefetching of Dataset.iter_batches ([doc](https://docs.ray.io/en/releases-2.4.0/data/api/doc/ray.data.DatasetIterator.iter_batches.html)), improving performance by fetching data in parallel while the main thread continues processing, thus reducing waiting time. * Support reading SQL databases ([doc](https://docs.ray.io/en/releases-2.4.0/data/creating-datasets.html#reading-from-sql-databases)), enabling users to seamlessly integrate relational databases into their _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.4.0]_ ### 2.3.1 — Ray-2.3.1 - Date: 2023-03-27 - Version: 2.3.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.3.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.3.1 - **changed** — Support different number of blocks/rows per block in zip() - **changed** — Revert serve run to use Ray Client instead of Ray Jobs - **fixed** — Fix issue with max_concurrent_queries being ignored when autoscaling - **fixed** — Write Ray address even if Ray node is started with --block - **fixed** — Fix Ray on Spark running on layered virtualenv python environment - **fixed** — Fix disk metric showing double the actual value The Ray 2.3.1 patch release contains fixes for multiple components: ##### Ray Data Processing * Support different number of blocks/rows per block in `zip()` (https://github.com/ray-project/ray/pull/32795) ##### Ray Serve * Revert `serve run` to use Ray Client instead of Ray Jobs (https://github.com/ray-project/ray/pull/32976) * Fix issue with `max_concurrent_queries` being ignored when autoscaling (https://github.com/ray-project/ray/pull/32772 and https://github.com/ray-project/ray/pull/33022) ##### Ray Core * Write Ray address even if Ray node is started with `--block` (https://github.com/ray-project/ray/pull/32961) * Fix Ray on Spark running on layered virtualenv python environment (https://github.com/ray-project/ray/pull/32996) ##### Dashboard * Fix disk metric showing double the actual value (https://github.com/ray-project/ray/pull/32674) ### 2.3.0 — Ray-2.3.0 - Date: 2023-02-24 - Version: 2.3.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.3.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.3.0 - **added** — Streaming backend for Ray Datasets in Developer Preview for terabyte-scale ML inference and training workloads - **added** — New Information Architecture for Ray dashboard organized around user personas and workflows in Beta - **added** — Ray-on-Spark is now available in Preview to launch Ray clusters on Databricks and Spark clusters - **added** — Add set_preprocessor method to Checkpoint - **added** — Introduce DatasetIterator - **added** — Add per-epoch preprocessor - **added** — Add TorchVisionPreprocessor - **added** — Persist Dataset statistics automatically to log file - **added** — Add zero-copy batch API for ds.map_batches() - **changed** — Rename Keras callback and its parameters to be more descriptive - **changed** — Rename WandB save_checkpoints to upload_checkpoints - **changed** — Rename DLPredictor.call_model tensor parameter to inputs - **changed** — Lazy execution by default for Ray Datasets - **changed** — Introduce streaming execution backend - **changed** — Use Ray actors instead of multiprocessing for WandbLoggerCallback - **changed** — Change default NCCL_SOCKET_IFNAME to blacklist veth - **changed** — Change ScalingConfig to be optional for DataParallelTrainers if already in Tuner param_space - **deprecated** — Deprecate MlflowTrainableMixin in favor of setup_mlflow() function - **fixed** — Fix and improve support for HDFS remote storage - **fixed** — Fix Imputer failing with categorical dtype #### Release Highlights * The streaming backend for Ray Datasets is in Developer Preview. It is designed to enable terabyte-scale ML inference and training workloads. Please contact us if you'd like to try it out on your workload, or you can find the preview guide here: https://docs.google.com/document/d/1BXd1cGexDnqHAIVoxTnV3BV0sklO9UXqPwSdHukExhY/edit * New Information Architecture (**Beta**): We’ve restructured the [Ray dashboard](https://docs.ray.io/en/master/ray-core/ray-dashboard.html) to be organized around user personas and workflows instead of entities. * Ray-on-Spark is now available (Preview)!: You can launch Ray clusters on Databricks and Spark clusters and run Ray applications. Check out the [documentation](https://docs.ray.io/en/releases-2.3.0/cluster/vms/user-guides/community/spark.html) to learn more. #### Ray Libraries ##### Ray AIR 💫Enhancements: * Add `set_preprocessor` method to `Checkpoint` (#31721) * Rename Keras callback and its parameters to be more descriptive (#31627) * Deprecate MlflowTrainableMixin in favor of setup_mlflow() function (#31295) * W&B * Have train_loop_config logged as a config (#31901) * Allow users to exclude config values with WandbLoggerCallback (#31624) * Rename WandB `save_checkpoints` to `upload_checkpoints` (#31582) * Add hook to get project/group for W&B integration (#31035, 31643) * Use Ray actors instead of multiprocessing for WandbLoggerCallback (#30847) * Update `WandbLoggerCallback` example (#31625) * Predictor * Place predictor kwargs in object store (#30932) * Delegate BatchPredictor stage fusion to Datasets (#31585) * Rename `DLPredictor.call_model` `tensor` parameter to `inputs` (#30574) * Add `use_gpu` to `HuggingFacePredictor` (#30945) * Checkpoints * Various `Checkpoint` improvements (#30948) * Implement lazy checkpointing for same-node case (#29824) * Automatically strip "module." from state dict (#30705) * Allow user to pass model to `TensorflowCheckpoint.get_model` (#31203) 🔨 Fixes: * Fix and improve support for HDFS remote storage. (#31940) * Use specified Preprocessor configs when using stream API. (#31725) * Support nested Chain in BatchPredictor (#31407) 📖Documentation: * Restructure API References (#32535) * API Deprecations (#31777, #31867) * Various fixes to docstrings, documentation, and examples (#30782, #30791) 🏗 Architecture refactoring: * Use NodeAffinitySchedulingPolicy for scheduling (#32016) * Internal resource management refactor (#30777, #30016) ##### Ray Data Processing 🎉 New Features: * Lazy execution by default (#31286) * Introduce streaming execution backend (#31579) * Introduce DatasetIterator (#31470) * Add per-epoch preprocessor (#31739) * Add TorchVisionPreprocessor (#30578) * Persist Dataset statistics automatically to log file (#30557) 💫Enhancements: * Async batch fetching for map_batches (#31576) * Add informative progress bar names to map_batches (#31526) * Provide an size bytes estimate for mongodb block (#31930) * Add support for dynamic block splitting to actor pool (#31715) * Improve str/repr of Dataset to include execution plan (#31604) * Deal with nested Chain in BatchPredictor (#31407) * Allow MultiHotEncoder to encode arrays (#31365) * Allow specify batch_size when reading Parquet file (#31165) * Add zero-copy batch API for `ds.map_batches()` (#30000) * Text dataset should save texts in ArrowTable format (#30963) * Return ndarray dicts for single-column tabular datasets (#30448) * Execute randomize_block_order eagerly if it's the last stage for ds.schema() (#30804) 🔨 Fixes: * Don't drop first dataset when peeking DatasetPipeline (#31513) * Handle np.array(dtype=object) constructor for ragged ndarrays (#31670) * Emit warning when starting Dataset execution with no CPU resources available (#31574) * Fix the bug of eagerly clearing _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.3.0]_ ### 2.2.0 — Ray-2.2.0 - Date: 2022-12-13 - Version: 2.2.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.2.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.2.0 - **added** — Ray Jobs API is now generally available for submitting locally developed applications to a remote Ray Cluster - **changed** — Ray Dashboard now shows CPU flame graphs of Ray workers and new metrics for memory usage - **changed** — Out-Of-Memory (OOM) Monitor is now enabled by default - **changed** — Ray Data dynamic block splitting is now enabled by default to address out-of-memory and performance issues with large files - **added** — Add NumPy first path for Torch and TensorFlow Predictors in Ray AIR - **fixed** — Fix MLflow database integrity error in Ray AIR - **fixed** — Fix ResourceChangingScheduler dropping PlacementGroupFactory args in Ray AIR - **fixed** — Fix bug passing 'raise' to FailureConfig in Ray AIR - **fixed** — Fix reserved CPU warning if no CPUs are used in Ray AIR - **deprecated** — Deprecate Wandb mixin in Ray AIR - **deprecated** — Deprecate Checkpoint.to_object_ref and Checkpoint.from_object_ref in Ray AIR - **added** — Add select_columns() to select a subset of columns in Ray Data - **added** — Add write_tfrecords() to write TFRecord files in Ray Data - **added** — Support MongoDB data source in Ray Data - **added** — Add from_torch() to create dataset from Torch dataset in Ray Data - **added** — Add from_tf() to create dataset from TensorFlow dataset in Ray Data - **added** — Allow to set batch_size in BatchMapper in Ray Data - **added** — Support read/write from/to local node file system in Ray Data - **fixed** — Fix iter_batches() to not return empty batch in Ray Data - **added** — Add MosaicTrainer in Ray Train #### Release Highlights Ray 2.2 is a stability-focused release, featuring stability improvements across many Ray components. * [Ray Jobs API](https://docs.ray.io/en/releases-2.2.0/cluster/running-applications/job-submission/index.html#ray-jobs-api) is now GA. The Ray Jobs API allows you to submit locally developed applications to a remote Ray Cluster for execution. It simplifies the experience of packaging, deploying, and managing a Ray application. * [Ray Dashboard](https://docs.ray.io/en/releases-2.2.0/ray-core/ray-dashboard.html#ray-dashboard) has received a number of improvements, such as the ability to see cpu flame graphs of your Ray workers and new metrics for memory usage. * The [Out-Of-Memory (OOM) Monitor](https://docs.ray.io/en/releases-2.2.0/ray-core/scheduling/ray-oom-prevention.html) is now enabled by default. This will increase the stability of memory-intensive applications on top of Ray. * [Ray Data] we’ve heard numerous users report that when files are too large, Ray Data can have out-of-memory or performance issues. In this release, we’re enabling [dynamic block splitting](https://docs.ray.io/en/releases-2.2.0/data/dataset-internals.html#execution-memory) by default, which will address the above issues by avoiding holding too much data in memory. #### Ray Libraries ##### Ray AIR 🎉 New Features: * Add a NumPy first path for Torch and TensorFlow Predictors (#28917) 💫Enhancements: * Suppress "NumPy array is not writable" error in torch conversion (#29808) * Add node rank and local world size info to session (#29919) 🔨 Fixes: * Fix MLflow database integrity error (#29794) * Fix ResourceChangingScheduler dropping PlacementGroupFactory args (#30304) * Fix bug passing 'raise' to FailureConfig (#30814) * Fix reserved CPU warning if no CPUs are used (#30598) 📖Documentation: * Fix examples and docs to specify batch_format in BatchMapper (#30438) 🏗 Architecture refactoring: * Deprecate Wandb mixin (#29828) * Deprecate Checkpoint.to_object_ref and Checkpoint.from_object_ref (#30365) ##### Ray Data Processing 🎉 New Features: * Support all PyArrow versions released by Apache Arrow (#29993, #29999) * Add `select_columns()` to select a subset of columns (#29081) * Add `write_tfrecords()` to write TFRecord files (#29448) * Support MongoDB data source (#28550) * Enable dynamic block splitting by default (#30284) * Add `from_torch()` to create dataset from Torch dataset (#29588) * Add `from_tf()` to create dataset from TensorFlow dataset (#29591) * Allow to set `batch_size` in `BatchMapper` (#29193) * Support read/write from/to local node file system (#29565) 💫Enhancements: * Add `include_paths` in `read_images()` to return image file path (#30007) * Print out Dataset statistics automatically after execution (#29876) * Cast tensor extension type to opaque object dtype in `to_pandas()` and `to_dask()` (#29417) * Encode number of dimensions in variable-shaped tensor extension type (#29281) * Fuse AllToAllStage and OneToOneStage with compatible remote args (#29561) * Change `read_tfrecords()` output from Pandas to Arrow format (#30390) * Handle all Ray errors in task compute strategy (#30696) * Allow nested Chain preprocessors (#29706) * Warn user if missing columns and support `str` exclude in `Concatenator` (#29443) * Raise ValueError if preprocessor column doesn't exist (#29643) 🔨 Fixes: * Support custom resource with remote args for `random_shuffle()` (#29276) * Support custom resource with remote args for `random_shuffle_each_window()` (#29482) * Add PublicAPI annotation to preprocessors (#29434) * Tensor extension column concatenation fixes (#29479) * Fix `iter_batches()` to not return empty batch (#29638) * Change `map_batches()` to fetch input blocks on-demand (#29289) * Change `take_all()` to not accept limit argument (#29746) * Convert between block and batch correctly for `map_groups()` (#30172) * Fix `stats()` call causing Dataset _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.2.0]_ ### 2.1.0 — Ray-2.1.0 - Date: 2022-11-08 - Version: 2.1.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.1.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.1.0 - **added** — Ray Datasets read_images() API for loading image data - **added** — Ray Datasets read_tfrecords() API to read TFRecord files - **added** — Numpy-based API for user-defined functions in Preprocessor - **added** — Ray Serve support for gRPC endpoint (alpha release) - **added** — RLlib decision transformer (DT) algorithm - **added** — RLlib on_episode_created() callback hook - **added** — Learning rate schedule to SimpleQ and PG algorithms - **added** — Ray OOM prevention (alpha release) - **added** — Support dynamic generators as task return values - **added** — Dashboard time series metrics support - **added** — Dashboard export configuration files for Prometheus or Grafana - **added** — Dashboard new progress bar in job detail view - **added** — NumPy batch format to Preprocessor and BatchMapper - **added** — Ragged tensor extension type - **added** — KBinsDiscretizer Preprocessor - **added** — FullyShardedDataParallel support to TorchTrainer - **added** — Tuner.get_results() to retrieve results after restore - **changed** — Auto-enable GPU for Predictors - **changed** — Deprecate Checkpoint.to_object_ref and Checkpoint.from_object_ref - **changed** — Deprecate legacy train/tune functions in favor of Session #### Release Highlights * Ray AI Runtime (AIR) * Better support for Image-based workloads. * Ray Datasets `read_images()` API for loading data. * Numpy-based API for user-defined functions in Preprocessor. * Ability to read TFRecord input. * Ray Datasets `read_tfrecords()` API to read TFRecord files. * Ray Serve: * Add support for gRPC endpoint (alpha release). Instead of using an HTTP server, Ray Serve supports gRPC protocol and users can bring their own schema for their use case. * RLlib: * Introduce decision transformer (DT) algorithm. * New hook for callbacks with `on_episode_created()`. * Learning rate schedule to SimpleQ and PG. * Ray Core: * Ray [OOM prevention](https://docs.ray.io/en/master/ray-core/scheduling/ray-oom-prevention.html) (alpha release). * Support dynamic generators as task return values. * Dashboard: * Time series metrics support. * Export configuration files can be used in Prometheus or Grafana instances. * New progress bar in job detail view. #### Ray Libraries ##### Ray AIR 💫Enhancements: * Improve readability of training failure output (#27946, #28333, #29143) * Auto-enable GPU for Predictors (#26549) * Add ability to create TorchCheckpoint from state dict (#27970) * Add ability to create TensorflowCheckpoint from saved model/h5 format (#28474) * Add attribute to retrieve URI from Checkpoint (#28731) * Add all allowable types to WandB Callback (#28888) 🔨 Fixes: * Handle nested metrics properly as scoring attribute (#27715) * Fix serializability of Checkpoints (#28387, #28895, #28935) 📖Documentation: * Miscellaneous updates to documentation and examples (#28067, #28002, #28189, #28306, #28361, #28364, #28631, #28800) 🏗 Architecture refactoring: * Deprecate Checkpoint.to_object_ref and Checkpoint.from_object_ref (#28318) * Deprecate legacy train/tune functions in favor of Session (#28856) ##### Ray Data Processing 🎉 New Features: * Add read_images (#29177) * Add read_tfrecords (#28430) * Add NumPy batch format to Preprocessor and `BatchMapper` (#28418) * Ragged tensor extension type (#27625) * Add KBinsDiscretizer Preprocessor (#28389) 💫Enhancements: * Simplify to_tf interface (#29028) * Add metadata override and inference in `Dataset.to_dask()` (#28625) * Prune unused columns before aggregate (#28556) * Add Dataset.default_batch_format (#28434) * Add partitioning parameter to read_ functions (#28413) * Deprecate "native" batch format in favor of "default" (#28489) * Support None partition field name (#28417) * Re-enable Parquet sampling and add progress bar (#28021) * Cap the number of stats kept in StatsActor and purge in FIFO order if the limit exceeded (#27964) * Customized serializer for Arrow JSON ParseOptions in read_json (#27911) * Optimize groupby/mapgroups performance (#27805) * Improve size estimation of image folder data source (#27219) * Use detached lifetime for stats actor (#25271) * Pin _StatsActor to the driver node (#27765) * Better error message for partition filtering if no file found (#27353) * Make Concatenator deterministic (#27575) * Change FeatureHasher input schema to expect token counts (#27523) * Avoid unnecessary reads when truncating a dataset with `ds.limit()` (#27343) * Hide tensor extension from UDFs (#27019) * Add __repr__ to AIR classes (#27006) 🔨 Fixes: * Add upper bound to pyarrow version check (#29674) (#29744) * Fix map_groups to work with different output type (#29184) * read_csv not filter out files by default (#29032) * Check columns when adding rows to TableBlockBuilder (#29020) * Fix the peak memory usage calculation (#28419) * Change sampling to use same API as read Parquet (#28258) * Fix column assignment in Concatenator for Pandas 1.2. (#27531) * Doing partition filtering in reader constructor (#27156) * Fix split ownership (#27149) 📖Documentation: * Clarify dataset transformation. (#28482) * Update map_batches do _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.1.0]_ ### 2.0.1 — Ray-2.0.1 - Date: 2022-10-22 - Version: 2.0.1 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.0.1 - Permalink: https://whatsnew.fyi/product/ray/releases/2.0.1 - **changed** — Upgrade grpcio version to 1.32 - **changed** — Upgrade redis version to 7.0.5 - **fixed** — Fix segfault when using runtime environments - **changed** — Increase RPC timeout for dashboard - **fixed** — Set correct path when using python -m - **fixed** — Fix autoscaling for 0 CPU head node - **fixed** — Allow code in private remote Git URIs to be imported in Serve - **added** — Allow host and port in Serve config - **added** — RLlib evaluation supports asynchronous rollout - **fixed** — Fix hang during checkpoint synchronization in Tune - **fixed** — Fix trial restoration from different IP in Tune - **fixed** — Fix custom synchronizer serialization in Tune - **changed** — Replace deprecated name option with task_id in Workflows The Ray 2.0.1 patch release contains dependency upgrades and fixes for multiple components: - Upgrade grpcio version to 1.32 ([#28025](https://github.com/ray-project/ray/pull/28025)) - Upgrade redis version to 7.0.5 ([#28936](https://github.com/ray-project/ray/pull/28936)) - Fix segfault when using runtime environments ([#28409](https://github.com/ray-project/ray/pull/28409)) - Increase RPC timeout for dashboard ([#28330](https://github.com/ray-project/ray/pull/28330)) - Set correct path when using `python -m` ([#28140](https://github.com/ray-project/ray/pull/28140)) - [Autoscaler] Fix autoscaling for 0 CPU head node ([#26813](https://github.com/ray-project/ray/pull/26813)) - [Serve] Allow code in private remote Git URIs to be imported ([#28250](https://github.com/ray-project/ray/pull/28250)) - [Serve] Allow `host` and `port` in Serve config ([#27026](https://github.com/ray-project/ray/pull/27026)) - [RLlib] Evaluation supports asynchronous rollout (single slow eval worker will not block the overall evaluation progress). ([#27390](https://github.com/ray-project/ray/pull/27390)) - [Tune] Fix hang during checkpoint synchronization ([#28155](https://github.com/ray-project/ray/pull/28155)) - [Tune] Fix trial restoration from different IP ([#28470](https://github.com/ray-project/ray/pull/28470)) - [Tune] Fix custom synchronizer serialization ([#28699](https://github.com/ray-project/ray/pull/28699)) - [Workflows] Replace deprecated `name` option with `task_id` ([#28151](https://github.com/ray-project/ray/pull/28151)) ### 2.0.0 — Ray-2.0.0 - Date: 2022-08-23 - Version: 2.0.0 - Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.0.0 - Permalink: https://whatsnew.fyi/product/ray/releases/2.0.0 - **added** — Ray AIR, a scalable and unified toolkit for ML applications, is now in Beta - **added** — Ray now supports natively shuffling 100TB or more of data with the Ray Datasets library - **added** — KubeRay, a toolkit for running Ray on Kubernetes, is now in Beta - **added** — Ray Serve's Deployment Graph API is a new and easier way to build, test, and deploy an inference graph of deployments - **added** — BatchPredictors now have support for scalable inference on GPUs - **added** — All Predictors can now be constructed from pre-trained models - **added** — Add ImageFolderDatasource - **added** — Add the NumPy batch format for batch mapping and batch consumption - **added** — Add iter_torch_batches() and iter_tf_batches() APIs - **added** — Add local shuffling API to iterators - **added** — Add drop_columns() API - **added** — Add randomize_block_order() API - **added** — Add random_sample() API - **added** — Add support for len(Dataset) - **added** — Add UDF passthrough args to map_batches() - **added** — Add Concatenator preprocessor - **changed** — ray.ml is renamed to ray.air - **changed** — ray.ml.preprocessors have been moved to ray.data - **changed** — ray.ml.trainers have been moved to ray.train - **changed** — ray.ml.predictors has been moved to ray.train #### Release Highlights Ray 2.0 is an exciting release with enhancements to all libraries in the Ray ecosystem. With this major release, we take strides towards our goal of making distributed computing scalable, unified, and open. Towards these goals, Ray 2.0 features new capabilities for unifying the machine learning (ML) ecosystem, improving Ray's production support, and making it easier than ever for ML practitioners to use Ray's libraries. **Highlights:** * [Ray AIR](https://docs.ray.io/en/releases-2.0.0/ray-air/getting-started.html), a scalable and unified toolkit for ML applications, is now in Beta. * ​​Ray now supports [natively shuffling 100TB or more of data](https://docs.ray.io/en/releases-2.0.0/data/performance-tips.html#enabling-push-based-shuffle) with the Ray Datasets library. * [KubeRay](https://docs.ray.io/en/releases-2.0.0/cluster/kubernetes/index.html), a toolkit for running Ray on Kubernetes, is now in Beta. This replaces the legacy Python-based Ray operator. * [Ray Serve’s Deployment Graph API ](https://docs.ray.io/en/releases-2.0.0/serve/model_composition.html#serve-model-composition)is a new and easier way to build, test, and deploy an inference graph of deployments. This is released as Beta in 2.0. A migration guide for all the different libraries can be found here: [Ray 2.0 Migration Guide](https://docs.google.com/document/d/12ODPbhEzeyDRUt8ehHDiKCFoxksPWJOUNEicGhNxtRg/edit#). #### Ray Libraries ##### Ray AIR Ray AIR is now in beta. Ray AIR builds upon Ray’s libraries to enable end-to-end machine learning workflows and applications on Ray. You can install all dependencies needed for Ray AIR via `pip install -u "ray[air]"`. 🎉 **New Features:** * Predictors: * BatchPredictors now have support for scalable inference on GPUs. * All Predictors can now be constructed from pre-trained models, allowing you to easily scale batch inference with trained models from common ML frameworks. * ray.ml.predictors has been moved to the Ray Train namespace (ray.train). * Preprocessing: New preprocessors and API changes on Ray Datasets now make feature processing easier to do on AIR. See the Ray Data release notes for more details. * New features for Datasets/Train/Tune/Serve can be found in the corresponding library release notes for more details. 💫 **Enhancements:** * Major package refactoring is included in this release. * ray.ml is renamed to ray.air. * ray.ml.preprocessors have been moved to ray.data. * train_test_split is now a new method of ray.data.Dataset (#27065) * ray.ml.trainers have been moved to ray.train (#25570) * ray.ml.predictors has been moved to ray.train. * ray.ml.config has been moved to ray.air.config (#25712). * Checkpoints are now framework-specific -- meaning that each Trainer generates its own Framework-specific Checkpoint class. See Ray Train for more details. * ModelWrappers have been renamed to PredictorDeployments. * API stability annotations have been added (#25485) * Train/Tune now have the same reporting and checkpointing API -- see the Train notes for more details (#26303) * ScalingConfigs are now Dataclasses not Dict types * Many AIR examples, benchmarks, and documentation pages were added in this release. The Ray AIR documentation will cover breadth of usage (end to end workflows across different libraries) while library-specific documentation will cover depth (specific features of a specific library). 🔨 **Fixes:** * Many documentation examples were previously untested. This release fixes those examples and adds them to the CI. * Predictors: * Torch/Tensorflow Predictors have correctness fixes (#25199, #25190, #25138, #25136) * Update `KerasCallback` to work with `TensorflowPredictor` (#26089) * Add streaming BatchPredictor support (#25693) * Add `predict_pandas` implementation (#25534) * Add `_predict_arrow` interface for Predictor ( _[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.0.0]_