Ray

Highlights

• Ray Data: We added fixes for several to_pandas regressions introduced in 2.56: an opt-out flag (RAY_DATA_ENABLE_ARROW_BACKED_PANDAS_CONVERSION) for Arrow-backed conversion, an int64/double[pyarrow] overflow crash on concatenation, and a TensorDtype.__from_arrow__ crash on empty tensor columns (#64793, #64794).
• Ray Core: We added early detection for system-slice memory pressure: the memory monitor now snapshots the user and system cgroup slices together and logs an error when the system slice exceeds reserved system memory, warning users to raise --system-reserved-memory before it causes node deaths (#64492).
• Ray Serve: We added protobuf 7 compatibility and a routing fix for LLM direct streaming, so body-aware routers like PrefixCacheAffinityRouter no longer hang when RAY_SERVE_LLM_ENABLE_DIRECT_STREAMING=1 (#64592, #64488).

Ray Data

🔨 Fixes

• Fixed two Arrow-backed to_pandas regressions: added DataContext.enable_arrow_backed_pandas_conversion as an opt-out, and reconciled divergent numeric column types before concatenation to avoid int64/double[pyarrow] overflow crashes (#64793, #64768).
• Fixed a TensorDtype.__from_arrow__ crash on zero-size tensor elements by using an explicit row count instead of numpy's -1 dimension inference (#64794, #64767).
• Fixed a crash in hash partition caused by read-only hash arrays (#64584, #64552, #64559).
• Nullified _input_dependencies in _get_args so exporting operator args no longer triggers an exponential sanitize_for_struct call chain over fused operators (#64412, #64316).

Ray Serve

🔨 Fixes

• Added protobuf >=7 compatibility to _proto_to_dict by binding to FieldDescriptor.is_repeated when the deprecated label attribute is absent (#64592, #64362).

Ray LLM

🔨 Fixes

• Fixed direct-streaming routing for body-aware routers: the ingress now parses the raw request body into a SimpleNamespace over routing-key fields (messages, prompt) so choose_replica receives the message body instead of raw bytes (#64488, #64328, #64326).

Ray RLlib

🔨 Fixes

• Upgraded the ONNX example from the retired MobileNet v1 to MobileNet v3 via torchvision and pinned onnxscript in the GPU/ml-build CI dep locks, fixing ONNX export failures (#64591, #64028, #64031, #64590, #64033).

Ray Core

💫 Enhancements

• The threshold memory monitor now snapshots both the user and system cgroup slices and logs an error when system-slice usage exceeds the reserved system memory, prompting users to raise --system-reserved-memory (#64492).

🔨 Fixes

• Disabled NCCL cuMem host buffer registration in CI pytests to stabilize GPU test runs (#64580, #64146).

Documentation

📖 Documentation

• Repointed the ASHA Tune example links to the renamed README document, fixing the fail_on_warning ReadTheDocs build on the release line (#64761, #64630).
• Updated the Python 3.10 CPU pip freeze dependency list for the Ray 2.56.0 release (#64447, #64357).

Highlights

• Ray Data Stability: In this Ray release, we've added a variety of stability improvements, including running multiple datasets in a cluster, adding automatic batch size selection to CPU-based map-batches, and default logical memory configuration to prevent OOMs. We've also tightened iter_batches stability by reducing hidden buffering and shutting down the executor when consumers exit early (#63660, #63682, #62949). This reduces object-store spilling for common training workloads
• Ray Serve: We re-architected Ray Serve LLM by decoupling request handling from token streaming response path (#62667, #62680, #62668, #62669, #63167), resulting in significant LLM serving performance improvements. We've also introduced new routing policies such as session-sticky routing via consistent hashing with ConsistentHashRouter (#62905, #63096, #62906) and CapacityQueueRouter (#62323) which is beneficial for supply-constrained workloads.
• Ray Core: We've added GPU-domain-aware placement groups using label locality (#61442, #61614, #62487, #62533). This enables placement groups to pack bundles onto nodes that share a ray.io/gpu-domain label instead of only packing at the single-node level. We've also added initial Kubernetes in-place pod resizing support for Autoscaler v2 (#55961, #62369, #62215), enabling Ray clusters to resize CPU and memory on existing worker pods before scaling out new pods.

Ray Data

🎉 New Features

• Support multiple datasets per cluster via subcluster labels and resource partitioning (#63331, #63375, #63982)
• Add Dataset.mix() public API and MixOperator for weighted dataset mixing (#63168, #62450)
• New DataSourceV2 framework: ParquetDatasourceV2, chunked reader, predicate splitting, listing/scanner infra (#63113, #63454, #63163, #62975, #63027, #62182)
• Add batch_size='auto' to map_batches to derive batch row count from target row batch size (#62648)
• Implement distributed upsert for Iceberg using task-based merge algorithm, preventing performance bottleneck on driver (#63482)
• Add include_row_hash to read_parquet (#61408)
• Add JAX data iterator (#61630)
• Expose flag to run read tasks on isolated worker processes via isolate_read_workers (#63490)
• Expose flag to set default logical memory for map operators via default_map_logical_memory_enabled (#63814)
• Support predicate pushdown for Lance format (#61400)
• Support per-partition start_offset and end_offset for read_kafka (#61620)
• Add obstore async download backend for download operator (#61735)
• Support UDF retries on transient exceptions (#63023)

💫 Enhancements

• Fix iter_batches spilling by replacing make_async_gen with iter_threaded and reducing buffered batches (#63660, #63682)
• Gate restore_original_order in iter_batches behind preserve_order (#63792)
• Convert drop_columns to a Project logical operator when input schema is known (#63813)
• Make ConcatAggregation and TurbopufferDatasink use polars for sorting (#61904)
• Boost and vectorize hash_partition with sort_indices, zero-copy slices, and pandas (#63498, #62757, #63152, #62587)
• Enable GPU_SHUFFLE in grouped_data.py (#62410)
• Eager StarExpr expansion, schema inference for non-black-box UDFs, and Expressions struct support (#63776, #63387, #62560)
• Make logging configurable via RAY_DATA_LOG_LEVEL and log RAY_DATA env vars at execution start (#63487, #63380)
• Display and track logical memory in progress bar (#63379)
• Honor compute= in filter(expr=...) and deprecate concurrency= (#63576)
• Enable filter pushdown through StreamingRepartition and read stage column-rename removal (#62347, #63384, #63582)
• Cache deserialized Arrow schemas in BlockMetadataWithSchema (#63462)
• Track scheduling-loop step duration (p50/p90/max), peak USS/object-store memory, and task block locality (#63586, #63345, #63489, #63418, #62249)
• Replace TaskDurationStats and Timer with DistributionTracker (#63488, #63530, #63825)
• Introduce BlockEntry on RefBundle in place of (ref, metadata) tuples (#63654)
• Pre-resolve filesystem in threaded download to avoid IMDS herd (#62898)
• Convert logical operators to frozen dataclasses and consolidate operator base/repr (#62593, #62568, #62400, #63137, #63140, #63108)
• Non-blocking default autoscaling coordinator and resource-aware auto-downscaling (#62725, #62574)
• Release pinned blocks after dataset execution and shut down executor on early DataIterator exit (#62456, #62949)
• Optimize local shuffle with incremental index and configurable compaction threshold (#62539)
• Speed up checkpoint filter and reduce memory usage (#60294)
• Preserve Arrow types through pandas roundtrip and reorder block columns by name before schema ops (#63017, #63582)
• Block pickle object columns when reading untrusted Parquet and gate unsafe WebDataset deserialization (#63470, #63469)
• Move backpressure escape hatch across all policies (#63539)
• Update pandas, modin, and pyarrow minimum versions (#62899)
• Add utilization monitoring and correct logical resource usage for ActorPool (#61987, #61528)
• Deprecate ConcurrencyCapBackpressurePolicy, DataIterator.to_torch, and pandas UDF batches (#63392, #62540, #61733)
• Rank actors per node in a heap and avoid re-exporting actor class via .options (#62309, #62722)
• read_delta reads from preconfigured pyarrow dataset (#61721)
• Include column name and target type in ArrowConversionError; reduce arrow conversion warning verbosity (#62407, #61486, #62521)
• Show external consumer bytes in verbose operator progress log (#63728)
• Disable DataSourceV2 by default after earlier enabling (#63674, #63326)

🔨 Fixes

• Rename subcluster label key from __subcluster__ to ray-subcluster (#63982)
• Fix get_or_create_stats_actor crash in Ray Client mode (#63402)
• Fix datasource pushdown crashes for generic UDFExpr filter predicates (#63781)
• Fix hash-shuffle aggregator memory estimation: metadata propagation, node-size clamp, column pruning (#63809)
• Fix CheckpointConfig FileNotFoundError on Azure Blob Storage (#63606)
• Fix silent credential drop for fsspec-S3 in download expression (#62897)
• Fix missing f-string prefix in _concatenate_extension_column (#62939)
• Fix HashAggregate duplicate group rows for AggregateFnV2 (#63066)
• Fix JSONL read retry with advanced file cursor (#63233)
• Fix read_parquet ArrowNotImplementedError for nested column types exceeding ~2GB row group (#61824)
• Fix read_parquet nested-type fallback and parquet scanner memory accumulation (#63175, #62745)
• Fix memory leak in DataIterator.to_torch() by switching to PyArrow (#60966)
• Fix ZipOperator freeing shared blocks via _split_at_indices (#62665)
• Fix concurrent writes race condition in write_parquet (#62377)
• Fix GPU shuffle output ordering when using ShuffleStrategy.GPU_SHUFFLE (#62351)
• Fix incorrect DatasetStat uuid propagation (#62255)
• Fix none issue when DATA_ENABLE_OP_RESOURCE_RESERVATION=False (#61718)
• Fix filesystem compatibility check for fsspec-wrapped PyFileSystem (#61850)
• Forward try_create_dir to pyarrow.dataset.write_dataset (#58302)
• Fix autoscaler bug blocking timely release of leased resources (#62592)
• Ensure consistent nan_is_null/nans-as-nulls semantics in encoder (#62623, #62618)
• Skip unconditional null strip in find_partition_index (#62594)
• V1 _split_predicate_by_columns correctness fix (#63176)
• Avoid importing cudf in _is_cudf_dataframe when cudf not loaded (#62302)
• Revert raw-modulo hash partition fast path (#63097)
• Remove tfx-bsl support from read_tfrecords (#63245)

📖 Documentation

• Document isolate_read_workers for read_parquet (#63816)
• Remove docs recommending increased object store memory proportion (#63389)
• Update docs minimum version for build_processor and "auto" batch size (#61757, #62790)
• Remove outdated limitation of DefaultClusterAutoscalerV2 and stale object-store-memory warnings (#62385, #62387)

Ray Serve

🎉 New Features:

• Add custom ingress request router app interfaces and HAProxy ingress dispatch path (#62680, #62668, #62669, #62667)
• Expose choose_replica/dispatch on deployment handles and AsyncioRouter with replica-side slot reservation (#63255, #63254, #63252)
• Introduce experimental round robin router and ConsistentHashRouter for session-sticky routing (#63238, #62906, #63096, #62905)
• Central capacity queue for token-based request routing via CapacityQueueRouter (#62323)
• Add experimental ray-haproxy support behind RAY_SERVE_EXPERIMENTAL_PIP_HAPROXY (#62589)
• Add deployment actor context API and broadcast API for deployment handles (#62532, #61472)
• Add ControllerOptions for configurable controller runtime_env (#63352)
• Make rolling update percentage configurable (#62160)
• Support per-request timeout and disconnect in HTTP proxy path (#62867)

💫 Enhancements:

• HAProxy stability improvements: wait for old workers before drain, redirect stdout/stderr, redispatch+retry-on, coalesce broadcasts, quarantine released ports (#63620, #63621, #63622, #63623, #63628)
• Bind direct ingress ports to 0.0.0.0 for cross-node HAProxy routing (#62515)
• HAProxy ingress request router metrics, enable splice by default, TCP_NODELAY default 1, optional retry knobs, RAY_SERVE_HAPROXY_STATS_PORT (#63356, #63531, #63353, #63415, #62979)
• Resolve bundled ray-haproxy binary before RAY_SERVE_HAPROXY_BINARY_PATH; HAProxy abspath env var (#63829, #62610)
• Replace socat subprocess with Python socket for HAProxy admin communication; bump HAProxy to avoid CVE-2025-11230 (#61897, #62585)
• Expose controller health metrics via /api/serve/applications/ API; add max_replicas_per_node to response (#63556, #63234)
• Run health check on user execution path to detect request-serving stalls (#61621)
• Mark widely-used APIs as stable (#62932)
• Retain recently-stopped replica logs in the dashboard (#63678)
• Add observability logs for pack scheduling decisions (#63603)
• Gate ingress request router body forwarding behind escape hatch (#63183)
• Avoid rolling replicas for no-op config overrides (#63034)
• Gate replica/deployment creation during shutdown (#62761)
• Defer PG creation for TPU Serve deployments to accelerator backend (#62941)
• Expose DeploymentStateManager APIs for controller access (#62950)
• Add tracing support for Windows and gRPC tracing improvements (#62821, #63833)
• Split node vs requested resources in deployment scheduler (#62778)
• Defer DEPLOYMENT_TARGETS broadcast while replicas are RECOVERING (#62751)
• Evict per-deployment LongPollHost state on deployment delete; enable logs when client stops its event loop (#62820, #63028)
• Add metrics: max replica processing latency, objref resolution latency, serve_autoscaling_target_ongoing_requests (#62381, #62355, #62421)
• Filter stale bootstrap observations from serve_long_poll_latency_ms (#62868)
• Retry build_serve_application task on failure (#62987)
• Scale down non-matching primary-label replicas first (#61488)
• Refactor internal autoscaling policy state extraction into a single helper (#62452)
• Catalog Ray Serve env vars (#62006)
• Remove or raise clear error for deprecated deployment items; remove deprecated DeploymentMode (#63548, #63510)

🔨 Fixes:

• Fix orphaned actors on controller crash during shutdown; drop and replace replicas surviving a controller crash without rank assignment (#62823, #63139)
• Fix deployment actors creating 15K OS threads for sync actor classes (#62661)
• Fix gang scheduling PG leak when deployment actors are starting (#62469)
• Fix app-level autoscaling policy state cross-deployment contamination and state loss for skipped deployments (#62484)
• Fix Serve autoscaling delay to use wall-clock time (#62144)
• Fix race condition in multiplex LRU cache update using move_to_end() (#62548)
• Normalize multiplexed model ID header to support proxy-transformed names (#61869)
• Fix AttributeError when request_router is None in update_deployment_config (#63180)
• Fix potential UnboundLocalError in ActorReplicaWrapper.check_stopped() (#63339)
• Fail loud when ingress request router dispatch fails (#63215)
• Fix stale _global_client cache across driver sessions (#62368)
• Fix start_metrics_pusher crash when deployment has record_autoscaling_stats but no autoscaling config (#62123)
• Fix high-cardinality namespace tag on long poll metrics (#62386)
• Fix Java long poll timeout serialization (#61875)
• Avoid destructor error when FastAPI ingress init fails (#62172)
• Avoid proxy readiness future timeout race (#62194)
• Avoid self-cause on non-gRPC replica exceptions (#62412)
• Fix HAProxy startup timeout propagation (#61752)
• Include ingress_request_router.lua.tmpl in package_data (#63145)
• Revert support for root_path parameter across uvicorn versions (#62529)

📖 Documentation:

• Add round robin and consistent hashing router documentation (#63636)
• Introduce gang scheduling documentation (#61737)
• Add deployment scope actor docs (#62735)
• Add Kuberay guide for RayService with HAProxy and High Throughput mode (#62408)
• Add Ray Serve office hours invite into documentation (#62176)

Ray Train

🎉 New Features

• Add LoggingConfig for configuring the ray.train logger on controller and workers (#61550)
• Allow DataParallelTrainer's train_fn to return data (#62021)
• Add async checkpointing/validation with Torch Lightning (#62370)

💫 Enhancements

• Report time spent syncing and transferring checkpoints to storage in ray.train.report(checkpoint) (#62027)
• Block until create_or_update_train_run completes on Train initialization (#63432)
• Implement DatasetManager (#63309)
• Forward label_selector to AutoscalingCoordinator (#63287)
• Add log line before launching training function (#62911)
• Allow contextlib.redirect_stdout() to bypass print redirect to logs (#61075)
• Add timeouts to validation functions of ray.train.report (#62916)
• ray.train.report does not hang across replica group restarts; Ray Train manages replica group restarts (#62651, #61475)
• Swallow RayTaskError during BackendSetupCallback shutdown (#63143)
• Improve JaxTrainer TPU multi-slice fault tolerance and reservation ergonomics (#62893)
• Export default data execution options (#62784)
• Consolidate Train run metadata sanitization and improve readability (#63182)
• Fix PlacementGroupCleaner race condition: drain queue before cleanup on controller death (#62754)
• Harden against unsafe pickle deserialization (#62807)
• Raise error when checkpoint is within experiment directory and delete_local_checkpoint_after_upload=True (#62555)
• Add timeout_s to ray.train.get_all_reported_checkpoints (#61761)
• Change remaining pytorch_lightning imports (#61291)
• Make controller resilient to errors in all lifecycle hooks (#60900)
• Remove Predictor from Train v1 (#63461)

🔨 Fixes

• Fix missing comma in DataBatchType Union type (#63872)
• Handle Arrow-backed pandas dtypes in LightGBM examples (#63427)
• Fix exclude_resources regression for V1 Train + V2 cluster autoscaler (#62827)
• Add missing %s to logger.debug (#63039)
• Increase get_actor timeout (#62516)

📖 Documentation

• Document S3-compatible storage (#63103)
• Add Azure Files to persistent storage docs (#63406)
• Uncomment Result.from_path in docs (#62887)
• Document how to tune async validation (#62227)
• Document why validation runs need unique names (#62224)

Ray Tune

💫 Enhancements

• Fix Tune search for Python 3.14 (#63575)
• Modernize AxSearch for Ax Platform 1.0.0+ (#60522)
• Use built-in inspect for argument capture (#60049)

🔨 Fixes

• Fix import count in CIFAR PyTorch tutorial (#62756)

Ray LLM

🎉 New Features

• Major Ray Serve LLM performance improvement with direct streaming (#63167, #63468, #63779)
• TPU support: Add topology field to LLMConfig for multi-host TPU support (#61906)
• Add per-host bundles default and fix fractional TPUs for TPUAccelerator (#63177)
• Enable Ray Serve LLM session-stickiness routing policy via RAY_SERVE_SESSION_ID_HEADER_KEY (#63362)

💫 Enhancements

• Upgrade vLLM to 0.22.0 (#63730, #63396, #62970, #62349)
• Co-locate DP rank 0 worker with advertised master address (#63803)
• Add pick-only fast path to AsyncioRouter for LLM ingress (#63517)
• Replace LLM ingress router replica selection with choose_replica; don't fetch LLMConfig from replicas at startup (#63280, #63065)
• Promote max_tasks_in_flight_per_actor to a first-class config field and adjust defaults (#63214)
• Validate accelerator_type against CPU-only configs; replace GPUType alias with AcceleratorType (#62139, #62978)
• Add rate-limiter for per-request traceback spam (#62440)
• Promote SGLang integration to user guide and move engine to _internal (#62570)
• Lazy-load batch stage/processor submodules and make boto3/botocore imports lazy (#62861, #62383)
• LLM telemetry bugfixes (#63782)

🔨 Fixes

• Fix flaky GPU-0 worker and NIXL port collisions (#63810)
• Fix P/D direct streaming OpenAI routing (#63679)
• Remove guided_decoding, truncate_prompt_tokens, build_llm_processor (#63569)
• Fix misleading ImportError when vLLM is installed but fails to import (#63305)
• Fix max_pending_requests default to track vLLM's GPU-dependent max_num_seqs (#62918)
• Fix HF config loading for models with custom rope_scaling (#62464)
• Wait for request router init in LLMRouter constructor (#63206)
• Materialize chat completion message content in sanitizer (#63119)
• Fix lora_request not forwarded to vLLM engine + add regression tests (#62609)
• Fix SGLangEngineProcessor telemetry for trust_remote_code models (#62102)
• Fix TOKENIZER_ONLY downloads missing chat_template for S3-backed models (#62121)
• Fix SGLang chat tokenize to respect add_generation_prompt (#61688)
• Fix bool serialization in benchmark_vllm CLI builder (#63516)

📖 Documentation

• Document multimodal pixel-budget gotchas and vLLM compatibility (#63593)
• Add tokenization disaggregation documentation (#62494)
• Add benchmark docs and refactor into submodules (#62204)
• Remove VLLM_USE_V1 from docs and examples (#63001)
• Fix wrong documented default for max_tasks_in_flight_per_actor (#62917)

Ray RLlib

🎉 New Features

• Add custom_resources_per_learner config and custom_resources_for_main_process to AlgorithmConfig (#63303, #62475)
• Add Importance Sampling APPO metrics to the torch learner (#63675)

💫 Enhancements

• Put only one copy of weights into the object store (#63529)
• Handle the all-evaluation-workers-unhealthy case uniformly across modes (#63128)
• Stop IMPALA/APPO learner thread gracefully to avoid misleading error messages (#62763)
• Improve invalid input error messages (#62324)

🔨 Fixes

• Fix two substantial edge cases in PPO's value target calculation (#59958)
• Fix EnvRunner crash loops (#62884)
• Fix extra model outputs hanging val indexing (#62960)
• Fix ValueError in MultiAgentEpisode.get_rewards() when an agent is inactive for all requested env steps (#62907)
• Preserve Torch optimizer param-group scalar types on restore (#61937)
• Fix wrong assert variable in _update_env_seed_if_necessary (#61823)
• Maintain value in EMAStat (#63064)

📖 Documentation

• Clarify extra model output docstrings (#63524)

Ray Core

🎉 New Features

• Add support for Furiosa AI NPU (#63035) and register_collective_backend API for custom collective backends (#60701)
• In-place pod resizing (IPPR) on Kubernetes 1.35: initial implementation and standalone KubeRay IPPR provider (#55961, #62369, #62215)
• Label locality support: GPU-domain-aware placement groups, autoscaler proto changes, and state API observability (#61442, #61614, #62487, #62533)
• Publish platform events via Ray Event Recorder and support single-event emission in the Python layer (#63329, #60858)
• Autoscaler v2: priority-based worker group selection (#62997) and noDriverTimeoutSeconds for KubeRay cluster termination (#63465)
• RDT: concurrent one-sided transfers for multiple ObjectRefs in ray.get (#61773), retry support (#62842), and NIXL memory deregistration via deregister_nixl_memory (#62341)
• Support .tar.gz archives for remote working_dir URIs (#62813)
• Add IPv6 localhost and all-interfaces support (#60023)

💫 Enhancements

• Resource isolation: event-based memory monitor, multi-memory-monitor factory, time-based group killing policy, idle-worker prioritization, system/user slice bounds, and OOM policy tuning (#62060, #62705, #62643, #62378, #62168, #63521, #63324, #63067, #62957)
• Compute per-component memory usage in MiB (#63932) and add host vs container memory distinction to memory panels (#63111)
• Consider cgroup limit when fetching CPU (#63685) and correct worker OOM score adjustment logic (#62470)
• Replace NodeAffinitySchedulingStrategy with Label Selector API when soft=False (#54940)
• Improve SlicePlacementGroup lifecycle and support explicit bundle_label_selector for TPUs (#63171); add TPU head resource for Ironwood TPU (#62786), chips_per_vm arg (#62526), and v6e single-host fixes (#62306)
• Batch placement group bundle removal RPCs (#63839); remove PG resource deduction from GCS in favor of resource broadcast (#63723)
• Migrate Raylet/GCS timing logic to a shared ClockInterface with a fake clock for testing (#62562, #62502, #62476)
• Refactor asio build targets and add IOContextMonitor; run GCS health check on io_service (#63042, #63166, #62608, #62374)
• Autoscaler v2 performance: skip serializations for debug logs (#63778); accept fractional resource values in request_resources (#63306)
• Reduce traffic: halve task arg pubsub by skipping redundant raylet pull (#62583), avoid extra memcpy when spilling fused objects (#63653), and resolve task dependencies synchronously when objects exist (#62561)
• Improve inspect_serializability messages and traversal context (#63501, #63373, #63258); better worker startup error messages (#63714)
• Warn when runtime_env package approaches upload size limit (#63404); harden zip extraction path containment (#63786, #62813)
• Include owner node ID in OwnerDiedError (#63727); add dependency info to taskspec debug string (#62316)
• Add unexpected worker failure metric and dashboard panel (#62297); group observability APIs in ray CLI help (#62748)
• Normalize OTel metric labels before Prometheus export (#63744) and retry/log when Prometheus queries fail (#63578); add GPU usage instance filter (#62214)
• Move observability and control-plane pubsub to dedicated services and rename InternalPubSub* to ControlPlanePubSub* (#62806, #63044, #62461)
• AMD GPU: replace rocm-smi ctypes binding with amd-smi Python interface (#62393); detect NVIDIA Blackwell consumer GPUs (#63322)
• Add task retry delay for ACTOR_UNAVAILABLE retries (#62330); improve State API filter key handling (#63638)
• Patch setproctitle to skip launch services IPC calls (#63366); add timeout for first redis probe (#63148)
• Clarify head node commands in ray up output (#63409); pass logging_config through Ray Client ray.init (#62192)
• Print subprocess log tails with exit codes on unexpected exit (#61905); add warning log when GPU profiling command times out (#63706)
• Add unique suffix to log filenames (#62365); disable profiling endpoints by default (#62531)
• Remove pydantic v1 support (#62716); update Starlette to v1.0.1 (#63722)
• Deprecate DAGNode.execute() (#63716); remove experimental _owner support for ray.put (#63520)

🔨 Fixes

• Fix ray.get hanging forever when an object's owner dies during pull (#63694); resolve ReferenceCounter race on WORKER_REF_REMOVED_CHANNEL (#60495)
• Fix resource leaks in subprocess management (#63878) and runtime_env cache not detecting changes in -r-referenced requirements files (#63403)
• Fix replica actor zombie process after GCS restart (#63764); fix actor creation race condition (#62994); fix actor state counter bug (#63647)
• Fix placement groups with label domain stuck on the infeasible queue (#62483); log status for failed PG PrepareResources/CommitResources (#62836)
• Fix env var expansion in ray job submit CLI via shlex.join (#63797) and --working-dir for local zip files and http:// URLs (#62843)
• Surface WebSocket close codes and errors in job log streaming (#63364); fix ray stop failing to terminate dashboard/runtime_env agents on Windows (#62428)
• Fix ray down not stopping Docker containers on worker nodes for local clusters (#62169); fix delayed/missing worker logs in Jupyter by flushing stdout/stderr (#63599)
• Fix Python log monitor handling for same-inode truncated files (#63720); avoid os.getcwd() on import by lazily evaluating scratch_dir (#63040)
• Fix accelerator detection on NVIDIA Blackwell consumer GPUs (#63322); avoid FabricManager stall on NVLink systems in GpuProfilingManager (#63312)
• Fix POSIX semaphore crash in experimental mutable objects (#62328); fix overflow on exponential backoff multiplication (#62366)
• Fix OOM kill message wrong threshold with resource isolation (#62948); fix OpenTelemetryMetricRecorder singleton init guard (#63081)
• Fix MarkFootprintAsBusy clearing saved idle state for unrelated items (#62588); fix HandleIsLocalWorkerDead for drivers (#62688)
• Fix AttributeError on trace in client mode (#62955); fix IndexError in legacy post-mortem debugging (#61479)
• Keep strong references to fire-and-forget asyncio tasks (#63291); validate JobConfig code_search_path type (#62499)
• Fix uv existence check in UVProcessor (#62818); fix invalid default stats factory in ClusterStatus (#62934)
• Fix autoscaler v2 instance_type_name in autoscaling state (#62101) and stopped-node metric double counting (#62026)
• Fix ReadOnlyProviderConfigReader max_workers counting bug (#62819); fix circular import in ray_print_logs thread (#63410)
• Fix wrong container in spill-fusion threshold check (#63605); avoid emitting idle worker failure for unregistered failed workers (#62789)
• Avoid return in finally block (Python 3.14 SyntaxWarning) (#63742); fix typos and replace type() checks with isinstance() (#62154)

📖 Documentation

• Add "bring your own transport" docs page for RDT (#60308); doc changes for label locality support (#62551)
• Fix misleading docstrings on drain_node APIs (#62942); update outdated description for max_direct_call_object_size (#63164)

Dashboard

🎉 New Features

• Add Platform Events module with K8s event ingestion/caching and frontend UI (#62314, #63332)
• Show TPU stats on the Cluster tab (#63774)

💫 Enhancements

• Add py-spy --idle and --subprocesses flags to profiling endpoints (#63852)
• Pass Grafana cluster filter to Serve metrics URLs (#63211)
• Show last data load time (#63618)
• Add Name column to Jobs view from job_name metadata (#62257)
• Mask password arguments in get_entrypoint_name() to prevent password exposure (#61995)

🔨 Fixes

• Fix TPU metrics (#63998)
• Guard against zero num_cpus in k8s_utils.cpu_percent (#63729)
• Fix invalid PromQL when global_filters is empty in Grafana dashboard generation (#63687)
• Fix unexpected log line details pop-up in log viewer UI (#62637)

Ray Wheels and Images

• Bumped the Ray version for the 2.56.0 release.
• Bumped the minimum Python version in pyproject.toml (#62569).
• Added TPU release images (#62113) and updated the TPU Docker image base dependencies (#63006).
• Added a torchft image for Torch trainer tests (#63361) and ran apt-get upgrade for slim base images (#62666).
• Numerous dependency lockfile and CI image updates (raydepsets migration, depset regeneration across core/ML/RLlib/docs/macOS CI images).

Documentation

• Established doc/redirects/current.yaml as the redirects source of truth with legacy-version 404 redirect coverage (#63367, #63880).
• Added an agent context guide for Ray documentation and an ipython3 lexer hook for notebook shell/magic cells (#63227, #63515).
• Added Sphinx /llms.txt and /llms-full.txt generation, excluding Jupyter notebooks (#63130, #63228).
• Upgraded doc toolchain: pydata-sphinx-theme 0.17.1, myst-nb 1.4.0, added sphinxext-opengraph, unpinned yanked tf-keras (#63344, #63360, #63343, #63358).
• Banned new .rst files under doc/source/ and added CI to skip RTD builds for PRs that don't touch docs (#63057, #63431).
• Added meta descriptions to ray-contribute pages and anonymized personal paths in Tune notebook outputs (#63832, #63464).
• Tune: updated deprecated sample_from examples to config-dict style and documented time_attr scheduler values (#63804, #32467).
• RLlib: clarified DQN hiddens as dueling-only, removed a broken parametric-actions link, fixed broken doc links (#43051, #54671, #47146).
• Ray Data: added a map_batches shuffle section, streaming generator docs, and fixed a broken README link (#62576, #63791, #63412).
• Ray Train: documented iter_jax_batches for JaxTrainer and updated TPU scaling config docs (#63294, #62584).
• Kubernetes/TPU: added a GKE Gateway ingress example, fixed the GKE TPU guide, and replaced deprecated example images (#63546, #63209, #63019).
• Added a RayCronJob quick-start guide and clarified KAI Scheduler RayJob submission modes (#62151, #61332).
• Added a Slurm guide for running Ray inside Docker containers (#63221).
• Documented AutoscalingConfig replica/target fields and corrected max_calls default docs (#48601, #63894).

Dependencies

This is the last Ray release to support the dependency versions listed below. For the 2.57.0 release, Ray will raise its minimum required versions for several core dependencies. If your environment pins any of these packages below the new minimums, plan to upgrade before moving to the next Ray release.

Most users on recent releases of these packages are unaffected

Thanks

Many thanks to all those who contributed to this release!

@khluu, @Krishnachaitanyakc, @leewyang, @ssam18, @christian-pinto, @Hyunoh-Yeo, @hango880623, @yuanzhuoyang1-bit, @marwan116, @aaronscalene, @tianyi-ge, @TriNguyen1208, @andrewsykim, @leonaIee, @OneSizeFitsQuorum, @AksodFlare, @limarkdcunha, @dayshah, @jade710, @pedrojeronim0, @dev-miro26, @DonPalius, @TimothySeah, @abrarsheikh, @nathon-lee, @prince8273, @Bye-legumes, @rayhhome, @Yunnglin, @spencer-p, @ryanaoleary, @herin049, @stephanie-wang, @liulehui, @slxswaa1993, @psaikaushik, @cyhapun, @tdat1465, @akyang-anyscale, @chenshi5012, @zzchun, @ryankert01, @EagleLo, @mzjp2, @justinvyu, @petern48, @YuangGao, @sjp611, @wingkitlee0, @AndySung320, @dstrodtman, @Accurio, @JasonLi1909, @peterjc123, @eicherseiji, @kyuds, @Chong-Li, @joaquinhuigomez, @IrvinFan, @XuQianJin-Stars, @AJamesPhillips, @harshit-anyscale, @claytonlin1110, @nhquana2, @Rruop, @win5923, @raulchen, @rohankmr414, @andrew-anyscale, @YoyinZyc, @doanxem99, @liujp, @dancingactor, @Evelynn-V, @SohamRajpure, @dragongu, @ShockYoungCHN, @ljstrnadiii, @WFY123wfy, @axreldable, @pseudo-rnd-thoughts, @H4ck2, @mvcb, @xinyuangui2, @edoakes, @ankushbbbr, @ps2181, @dominikkawka, @vinhuytran0810-cell, @siyuanfoundation, @MengjinYan, @Chronostasys, @jeffreywang88, @lalitc375, @sampan-s-nayak, @ArturNiederfahrenhorst, @srini047, @ChangyuWang, @adam360x, @Yicheng-Lu-llll, @thakoreh, @Aydin-ab, @manhld0206, @oab24413gmai, @ayushk7102, @tycao0338-cpu, @slfan1989, @myandpr, @rueian, @ans9868, @Ziy1-Tan, @elliot-barn, @as-jding, @daiping8, @robertnishihara, @MatthewCWeston, @Cursx, @laysfire, @karticam, @Mr-Neutr0n, @jjyao, @zent1n0, @aslonnie, @DenBuzz, @michael-pryor, @goanpeca, @nadongjun, @ronny-anyscale, @GoparapukethaN, @werkt, @carolynwang, @kamil-kaczmarek, @madiyar-wayve, @peterxcli, @pqkzzz, @Future-Outlier, @iamjustinhsu, @micah-yong-ai, @wxwmd, @owenowenisme, @sai-miduthuri, @lonexreb, @prassanna-ravishankar, @wanadzhar913, @kouroshHakha, @tobby168, @johntaylor-cell, @richabanker, @Kunchd, @vincere-mori, @vaishdho1, @wenhaozhao011-cmd, @bveeramani, @bittoby, @Phucvt123, @aschuh-hf, @RudrenduPaul, @xyuzh, @Sparks0219, @yancanmao, @eureka0928, @yuhuan130, @goutamvenkat-anyscale, @Zerui18, @machichima, @Lucas61000, @weimingdiit, @xi377266, @EmaFerrao, @awen11123, @Lawson-Darrow, @suppagoddo

• Fixes SSH connectivity issue in the ray-llm image (#62625 / #62718).
• Upgrade apt packages in slim base (#62666 / #62717).

Ray Data

🎉 New Features

• Add DataSourceV2 API with scanner/reader framework, file listing, and file partitioning (#61220, #61615, #61997)
• Support GPU shuffle with rapidsmpf 26.2 (#61371, #62062)
• Add Kafka datasink, migrate to confluent-kafka, support datetime offsets (#60307, #61284, #60909)
• Add Turbopuffer datasink (#58910)
• Add 2-phase commit checkpointing with trie recovery and load method (#61821, #60951)
• Queue-based autoscaling policy integrated with task consumers (#59548, #60851)
• Enable autoscaling for GPU stages (#61130)
• Expressions: add random(), uuid(), cast, and map namespace support (#59656, #60695, #59879)
• Add support for Arrow native fixed-shape tensor type (#56284)
• Support writing tensors to tfrecords (#60859)
• Add pathlib.Path support to read_* functions (#61126)
• Add cudf as a batch_format (#61329)
• Allow ActorPoolStrategy for read_datasource() via compute parameter (#59633)
• Introduce ExecutionCache for streamlined caching (#60996)
• Support strict=False mode for StreamingRepartition (#60295)
• Port changes from lance-ray into Ray Data (#60497)
• Enable PyArrow compute-to-expression conversion for predicate pushdown (#61617)
• Add vLLM metrics export and Data LLM Grafana dashboard (#60385)
• Include logical memory in resource manager scheduling decisions (#60774)
• Add monotonically increasing ID support (#59290)

💫 Enhancements

• Performance: cache _map_task args, heap-based actor ranking, actor pool map improvements (#61996, #62114, #61591)
• Optimize concat tables and PyArrow schema hashing (#61315, #62108)
• Reduce default DownstreamCapacityBackpressurePolicy threshold to 50% (#61890)
• Improve reproducibility for random APIs (#59662)
• Clamp batch size to fall within C++ 32-bit int range (#62242)
• Account for external consumer object store usage in resource manager budget (#62117)
• Make get_parquet_dataset configurable in number of fragments to scan (#61670)
• Consolidate schema inference and make all preprocessors implement SerializablePreprocessorBase (#61213, #61341)
• Disable hanging issue detection by default (#62405)
• Make execution callback dataflow explicit to prevent state leakage (#61405)
• Log DataContext in JSON format at execution start for traceability (#61150, #61428)
• Autoscaler: configurable traceback, Prometheus gauges, relaxed constraints (#62210, #62209, #61917, #61385)
• Add metrics for task scheduling time, output backpressure, and logical memory (#61192, #61007, #61436)
• Prevent operators from dominating entire shared object store budget (#61605)
• Eliminate generators to avoid intermediate state pinning (#60598)
• Default log encoding to UTF-8 on Windows (#61143)
• Remove legacy BlockList, locality_with_output, old callback API, PyArrow 9.0 checks (#60575, #61044, #62055, #61483)
• Upgrade to pyiceberg 0.11.0; cap pandas to <3 (#61062, #60406)
• Refactor logical operators to frozen dataclasses (#61059, #61308, #61348, #61349, #61351, #61364, #61481)
• Prevent aggregator head node scheduling (#61288)
• Add error for local:// paths with a zero-resource head node (#60709)

🔨 Fixes

• Fix RCE in Arrow extension type deserialization from Parquet (#62056)
• Fix StreamingSplitDataIterator.schema() (#62057)
• Fix ParquetDatasource handling of FileSystemFactory.inspect (#62065)
• Fix read_parquet file-extension filtering for versioned object-store URIs (#61376)
• Fix wide_schema_pipeline_tensors cloudpickle deserialization (#62149)
• Fix OpBufferQueue race condition (#60828)
• Fix scheduling metrics computation (#62031)
• Fix OneHotEncoder max_categories to use global top-k instead of per-partition (#60790)
• Fix ReservationOpResourceAllocator resource borrowing for ActorPoolMapOperator (#60882)
• Fix DatabricksUCDatasource schema() shadowing by schema string attribute (#61282)
• Fix AliasExpr structural equality to respect rename flag (#60711)
• Fix _align_struct_fields failure with unaligned scalar fields (#58364)
• Fix min_scheduling_resources fallback to incremental_resource_usage (#60997)
• Fix output backpressure unblocking sequence for terminal ops (#60798)
• Fix multi-input operator object store memory attribution (#61208)
• Fix reference cycle by moving to module scope (#61934)
• Fix autoscaler logging: reduce verbose output and move traceback to debug (#61989, #62126)
• Fix double counting ref_bundle + input_files (#61774)
• Replace on_exit hook with __ray_shutdown__ to fix UDF cleanup race (#61700)
• Prevent Limit from getting pushed past map_groups (#60881)
• Propagate schema in empty _shuffle_block to fix ColumnNotFound in chained left joins (#61507)
• Fix unclear metadata warning and incorrect operator name logging (#61380)
• Clamp rolling utilization averages to zero (#61543)
• Fix floating point errors in TimeWindowAverageCalculator (#61580)
• Remove default task-level timeout and clamp end_offset in Kafka datasource (#61476)
• Avoid redundant reads in train_test_split (#60274)
• Return None when no outputs have been produced (#62029)
• Replace bare raise with TypeError in string concatenation (#60795)

📖 Documentation

• Add job-level checkpointing documentation (#60921)
• Update exclude_resources docs for Train autoscaling changes (#61990)
• Add locality_with_output migration instructions (#61151)
• Document max_tasks_in_flight_per_actor vs max_concurrent_batches (#60477)
• Add missing MOD operation docs; improve ray.data.Datasource docs (#60803, #59654)
• Add polars usage instructions (#60029)

Ray Serve

🎉 New Features:

• Added end-to-end gRPC client and bidirectional streaming support, including public APIs, proxy handling, proto updates, and developer docs, so Serve apps can handle streaming workloads natively instead of building custom transport layers. (#60767, #60768, #60769, #60770, #60771)
• Introduced HAProxy-based serving with fallback proxy support and load-balancer tunables, giving operators a higher-throughput ingress path and more control over traffic behavior in production. (#60586, #61180, #61271, #61468, #61988)
• Added queue-based autoscaling for async inference and Taskiq-backed workloads, so scaling decisions can account for both HTTP in-flight load and queued tasks. (#59548, #60851, #60977, #61008)
• Rolled out gang scheduling support across validation, core scheduling, fault tolerance, downscaling, autoscaling, rolling updates, and migration, enabling coordinated multi-replica placement for tightly coupled workloads. (#60944, #61205, #61206, #61207, #61215, #61467, #61216, #61659)
• Introduced deployment-scoped actors with config/schema, lifecycle management, public API, and controller health checks, making it easier to run durable per-deployment sidecar-like logic inside Serve. (#61639, #61648, #61664, #61833, #62161)

💫 Enhancements:

• Added first-class tracing support for Serve, including inter-deployment gRPC propagation and richer streaming-path attributes, improving end-to-end observability across distributed request flows. (#61230, #61089, #61451)
• Expanded operational metrics with replica utilization, richer error labeling, and client IP logging in access logs, helping teams diagnose bottlenecks and user-impacting issues faster. (#60758, #61092, #60967)
• Improved autoscaling extensibility with class-based policies and policy_kwargs, so advanced users can package reusable autoscaling logic without custom forks. (#60964)
• Reduced controller overhead with broad algorithmic improvements (indexing, cache reuse, and avoiding repeated per-tick work), which improves scalability as deployment and replica counts grow. (#60810, #60829, #60830, #60838, #60842, #60843, #60844, #60832, #60806)
• Improved throughput-oriented operation controls by adding environment-based tuning and explicit throughput optimization logging, making performance behavior easier to configure and audit. (#60757, #62146)
• Upgraded Serve internals to Pydantic v2 and refined time-series aggregation behavior for more predictable metric accuracy under high load. (#61061, #61403)

🔨 Fixes:

• Fixed a direct-ingress shutdown bug where replicas could hang indefinitely while draining stuck requests, ensuring bounded shutdown behavior in failure scenarios. (#60754)
• Fixed HAProxy reliability issues, including config race conditions, draining guards, and platform compatibility edge cases, improving stability in production rollouts. (#61120, #60955)
• Fixed autoscaling correctness issues that could cause runaway scaling or delayed reactions, including feedback-loop regressions, streaming scale-down behavior, and wall-clock delay handling. (#61731, #61920, #62331, #61844, #60613)
• Fixed high-percentile latency regression in request routing and queue-length accounting, reducing tail-latency spikes under load. (#61755)
• Fixed replica-state and health-state edge cases during migration and ingress transitions, preventing false errors and unhealthy/healthy misreporting. (#60365, #61818, #62213)
• Fixed chained upstream actor-failure handling so request failures are attributed correctly and no longer hang when upstream deployments die mid-chain. (#61758, #62147)
• Fixed HTTP status classification for client disconnects after successful responses, improving accuracy of error-rate monitoring and alerting. (#61396)

📖 Documentation:

• Added AsyncInferenceAutoscalingPolicy documentation and clarified Serve performance guidance for HAProxy and inter-deployment gRPC use cases. (#61086, #61386)
• Updated scheduling and configuration docs, including replica scheduling guidance and a catalog of Serve environment variables, so operators can tune deployments with less guesswork. (#60922, #60807)
• Clarified multiplexing and async behavior docs (including model pre-warming constraints and request-cancel semantics) to prevent common integration mistakes. (#61842, #62280)

🏗 Architecture refactoring:

• Refactored deployment-state execution to skip unnecessary steady-state per-tick work, lowering control-loop churn and creating cleaner hooks for future scheduling logic. (#60840)
• Moved autoscaling metric aggregation into Cython-backed paths and added focused controller benchmarking, giving a stronger performance baseline for future Serve controller changes. (#58892, #61368)
• Simplified internal structure by migrating shared internals away from private modules and consolidating replica abstractions, reducing coupling and maintenance complexity. (#60849, #61363, #60198)

Ray Train

🎉 New Features

• Elastic training: core capability, user guide, release tests, multi-host TPU, telemetry (#60721, #61115, #61133, #61299, #61267)
• Add HF TRL (Transformer Reinforcement Learning) example (#61627)
• Add Tensor Parallel templates for DeepSpeed AutoTP and DTensor (#60160, #60158)
• Add status attribute to ReportedCheckpoint (#61684)
• Richer Train run metadata (#59186)
• Add timers for Train worker initialization (#60870)
• Configure torchft environment (#61156)

💫 Enhancements

• Register training resources with AutoscalingCoordinator in FixedScalingPolicy (#61703)
• Decouple datasets field from TrainRunContext (#61953)
• Log warning for checkpoint_upload_fn when slow (#61720)
• Fix StateManagerCallback to accept datasets explicitly (#62042)
• Make train run abortable during before_controller_shutdown (#61816)
• Graceful abort catches all RayActorError (#61375)
• Refactor checkpoint and sync_actor to use wait_with_logging (#61063)
• Unwrap UserExceptionWithTraceback in WorkerGroupError.worker_failures (#61153)

🔨 Fixes

• Fix v2 PlacementGroupCleaner zombie actor (#61756)
• Fix checkpoint paths for multinode run (#61471)
• Abort cancels validation tasks with deterministic resumption (#61510)
• Fix deepspeed finetune release test (#61266)

📖 Documentation

• Add section on async validation with experiment tracking (#62104)
• Add section on when to use async validation (#61702)

Ray Tune

💫 Enhancements

• Remove deprecated Logger interface and logger_creator (#61181)

🔨 Fixes

• Fix PBT trial order when NaN values are present (#57160)

Ray LLM

🎉 New Features

• Replace PDProxyServer with decode-as-orchestrator PD architecture (#62076)
• Introduce DP group fault tolerance for WideEP deployments (#61480)
• SGLang engine: streaming chat/completions, tokenize/detokenize, embeddings, multi-GPU TP/PP (#61236, #61446, #61159, #61201, #62221)
• Add bundle_per_worker config for simpler placement group setup (#59903)
• Separate Data and Serve LLM dashboards with improved panel visibility (#61037, #62069)

💫 Enhancements

• Promote Data LLM and Serve LLM APIs to beta (#61249, #62054, #62223)
• Upgrade vLLM to 0.16.0, 0.17.0, and 0.18.0 (#61389, #61598, #61952)
• Upgrade NIXL to v1.0.0 and fix tensor transport issues (#61991)
• Unify duplicated PlacementGroup config schemes (#62241)
• Decouple Serve LLM ingress from vLLM protocol models (#61931)
• Set download task num_cpus=0 to reduce contention on low-CPU machines (#61191)
• SGLangServer cleanup and replace format_messages_to_prompt with _build_chat_messages (#61117, #61372)

🔨 Fixes

• Fix duplicate data: [DONE] in streaming SSE responses (#62246)
• Fix enable_log_requests=False not forwarded to vLLM AsyncLLM (#60824)
• Fix OpenAiIngress scale-to-zero when all models set min_replicas=0 (#60836)
• Handle missing state attributes from vLLM's task-conditional init_app_state (#60812)
• Fix NIXL side channel host for cross-node P/D disaggregation (#60817)
• Fix trust_remote_code download (#60344)
• Avoid deprecated TRANSFORMERS_CACHE; treat HuggingFace config load failure as non-fatal (#60854)
• Fix sequential batch processing in SGLangServer (#61189)

📖 Documentation

• Update data parallel attention documentation (#61706)
• Add custom tokenizer example (#61098)
• Add C/C++ binaries incompatibility workaround (#62110)

Ray RLlib

💫 Enhancements

• Connector/batching optimizations: ndarray fast paths, direct env step pipeline, batch reuse (#61320, #61255, #61256, #61259, #61144)
• Unify default encoders for all algorithms (#60302)
• Toggle eval/train mode in TorchRLModule forward passes (#61985)
• Clean up offline prelearner and unit testing (#60632)
• Remove duplicate assignments in AlgorithmConfig (#61233)
• Remove legacy RLlib release tests (#59288)
• Add APPO example with Footsies environment (#59006)

🔨 Fixes

• Support custom eval functions returning zero eval_results, env_steps, or agent_steps (#61563)
• Fix PrioritizedEpisodeReplayBuffer bug (#60065)
• Fix missing LayerNorm in RLModuleSpec (#61025)
• Fix evaluation in parallel to training (#60777)
• Fix MultiAgentEpisode.env_t_to_agent_t (#60319)
• Fix default metric during eval (#61590)
• Fix incorrect log value of environment steps sampled/trained (#56599)
• Prevent torch_learner.py crash under parameter-freezing edge cases (#62158)

Ray Core

🎉 New Features

• Resource isolation: pressure-based memory monitor, time-based killing, cgroup constraints (#61361, #61323, #61097, #61210, #61297, #59365, #59368, #60752)
• IPPR: add ResizeRayletResourceInstances to GCS/Python client, schema/status models, KubeRay provider (#61654, #61666, #61803, #61814)
• Add PlatformEvent proto and placement group events in one-event framework (#61701, #60449)
• Add Nvidia B300 support (#60753)
• Add UV support for Ray Client mode (#60868)
• Add Percentile metric type backed by quadratic histogram (#61148)
• Expose fallback_strategy in TaskInfoEntry and ActorTableData (#60659)
• Add submission job proto changes (#60857)
• Add TPU util for ready multi-host slice count; simplify elastic TPU scaling (#61300, #62141)
• Introduce per-node level temp-dir (#60761)
• Make ray.put() generic: put(value: R) -> ObjectRef[R] (#60995)
• Add Python 3.14 support for recursion limit handling (#58459)

💫 Enhancements

• Upgrade cloudpickle to 3.1.2, gRPC to v1.58.0, protobuf to 3.20.3 (#60317, #61499, #60736)
• Multiple gRPC connections for improved object transfer throughput, enabled by default (#61121, #61440)
• Improve pg.ready() performance via async GCS RPC; fix deadlocks (#60657, #62086)
• RDT: non-torch transfers, PyTorch storage caching, metadata caching, NIXL agent reuse (#61081, #60999, #60689, #60602)
• Cache ActorHandle.__hash__ and fix __eq__ correctness (#61638)
• Cache find_gcs_addresses (#61065)
• Optimize worker listener thread (#61353)
• Eliminate Python GCS client from state manager get_all_node_info (#61232)
• Loosen restriction on worker thread count (#62279)
• Sequence in-order actor tasks per concurrency group instead of globally (#61082)
• Prioritize killing workers that occupy large memory in OOM killer (#60330)
• Cap exponential backoff attempt number to prevent integer overflow (#61003)
• Replace deprecated threading APIs (getName/setDaemon) (#62153)
• Improve error handling for @ray.remote/@ray.method with num_returns (#59286)
• Convert StopIteration on non-generator functions to RuntimeError (#60521)
• Surface warnings for scheduling rate limits slowing task ramp-up (#61004)
• Periodically reload service account tokens; use AuthenticationValidator in sync server (#60778, #60779)
• Remove support for local_mode (#60647)
• Allow matching worker_process_setup_hook on re-entry (#61473)
• Reduce default event aggregator buffer size to avoid OOM (#60826)
• Suppress autoscaler action logs for read-only provider (#61732)
• Lazy subscription to node changes on non-driver workers (#61118)
• Tighten export symbol allowlists to prevent non-ray symbol leakage (#61298)
• Approximate USS from memory_info instead of calling memory_full_info (#60000)
• Dedicated IO context for NodeManager and InternalKVManager (#61002)
• Print gRPC peer address on GCS HandleUnregisterNode/HandleDrainNode (#62226, #62112)

🔨 Fixes

• Fix task stuck when pop worker repeatedly fails (#60104)
• Fix bool env var parsing for RAY_CGRAPH_overlap_gpu_communication (#61421)
• Fix negative RUNNING task metric (#62070)
• Fix OnNodeDead to destroy all owned actors when owner node dies (#60669)
• Fix actor task queue blocked after cancelling head task (#60850)
• Fix TASK_PROFILE_EVENT aggregation for multiple phases (#61559)
• Fix double-counting in WorkerPool::WarnAboutSize() (#61246)
• Fix TaskLifecycleEvent.node_id using emitting node instead of executor (#61478)
• Fix publisher_id type mismatch in GCS pubsub (#61518)
• Fix dataclass.asdict with None in dashboard list_jobs API (#61033)
• Fix dashboard node head API dead node cache (#61185)
• Fix dashboard event agent for events without HTTP scheme (#60811)
• Fix Ray Actor typing for async methods (#60682)
• Fix autoscaler retry during k8s exceptions (#60658)
• Fix ReadOnlyProvider.terminate() signature mismatch (#62251)
• Fix set/get env races in OtlpGrpcMetricExporterOptions and metrics exporter init (#61034, #61281)
• Clean up node processes on version mismatch during ray start (#61837)
• Retry node discovery upon ray.init() (#61029)
• Ensure Node._node_labels initializes regardless of connect_only (#61618)
• Avoid reentrant locking in worker context (#61925)
• Java Local Mode type confusion with multiple Actor types (#61858)
• Recover from WrongClusterID on head restart (#60860)
• Fix Azure: do not delete shared MSI when tearing down clusters (#61811)
• Configure TLS/mTLS for OpenTelemetry OTLP gRPC exporter (#60745)

Dashboard

🎉 New Features

• Add Queued Blocks metric to Ray Data Dashboard (#61716)
• Add Logical Memory Usage panel (#60772)
• Add running tasks by node, update Ray Data active tasks panel (#61641)
• Add NIXL KV transfer metrics to Serve LLM Grafana dashboard (#60819)
• Add GPU power and temperature graphs (#60942)
• Support log links in Grafana dashboard (#60896)
• Support autoscaler v2 for cluster-level node metrics (#60504)
• Add middleware proxy for history server (#61295)
• Forward **kwargs through JobSubmissionClient to cluster info resolvers (#61902)

Ray Wheels and Images

• Upgrade Bazel from 6.5.0 to 7.5.0 (#61601)
• Bump torch to 2.7.0+cu128 and torchvision (#61328)
• Upgrade jackson-databind 2.16.1 -> 2.18.6 (GHSA-72hv-8253-57qq) (#61808)
• Upgrade CI containers from Ubuntu 20.04 to 22.04; Forge from clang-12 to clang-14 (#61533, #61662)
• Add CUDA 13 images for ray-llm/core-gpu and release test configs (#61497, #61637)
• Add py312+CUDA 12.9 and py312+CUDA 13 depsets for Ray LLM (#61116, #61149, #61496)
• Add TPU Docker images to CI build and publish pipeline (#61172, #61173, #61174, #61175)
• Add Python 3.14 to Linux wheel verification (#62127)
• Windows base build fix (#62415)
• Add build-image.sh and CLI for local Docker image builder (#61042, #61338)
• Support repeated execution of setup-dev.py (#61357)

Documentation

• Add Ray History Server user guide (#62030)
• Add RAY_BACKEND_LOG_JSON environment variable documentation (#59962)
• Add user guide for Ray token auth with Kubernetes RBAC (#61644)
• Add warning about token authentication in untrusted networks (#62248)
• KubeRay: prerunning deadline docs, version 1.6.0 refs, GKE/cgroups cross-refs (#61552, #61865, #62140)
• Use RayCluster name as ServiceAccount name for RBAC authentication (#61785)
• Remove outdated note on labels in local RayCluster (#61719)
• List TPUs as fully tested/supported (#61634)
• Restructure development.rst with image build, wheel paths, and cross-references (#61500, #61501, #61504, #61596)
• Remove incorrect warning for placement groups (#61176)
• Add multi-agent A2A example (#61193)
• Add object spill internal doc (#60930)

Thanks

Many thanks to all those who contributed to this release!

@justinyeh1995, @marwan116, @jddqd, @MkDev11, @mjd3, @XuQianJin-Stars, @elliot-barn, @DeborahOlaboye, @aaronscalene, @rayhhome, @ayushk7102, @bj-son, @nadongjun, @Daraan, @xinyuangui2, @Sparks0219, @justinvyu, @suppagoddo, @akyang-anyscale, @ambicuity, @Aydin-ab, @mickeyyliu, @MatthewCWeston, @vaishdho1, @jinbum-kim, @eicherseiji, @kouroshHakha, @karticam, @JasonLi1909, @ArturNiederfahrenhorst, @moktamd, @nrghosh, @dragongu, @andrewsykim, @mgchoi239, @ruoliu2, @harshit-anyscale, @Chong-Li, @pseudo-rnd-thoughts, @lee1258561, @khluu, @daiping8, @SolitaryThinker, @jonalee99, @yancanmao, @SohamRajpure, @rueian, @VitaliyEroshin, @Future-Outlier, @nehiljain, @JiangJiaWei1103, @Yicheng-Lu-llll, @KaisennHu, @jeffreywang-anyscale, @aslonnie, @alanwguo, @machichima, @limarkdcunha, @codope, @sampan-s-nayak, @kyuds, @thjung123, @abrarsheikh, @wingkitlee0, @preneond, @7ckingBest, @slfan1989, @win5923, @kaori-seasons, @israbbani, @andrew-anyscale, @zestze, @owenowenisme, @edoakes, @laysfire, @pushpavanthar, @tohtana, @leewyang, @liulehui, @Hyunoh-Yeo, @eureka0928, @ryanaoleary, @947132885, @Kunchd, @simonsays1980, @dpj135, @bveeramani, @raulchen, @Partth101, @dubin555, @richabanker, @bittoby, @sai-miduthuri, @RedGrey1993, @kamil-kaczmarek, @TimothySeah, @myandpr, @rishic3, @justinrmiller, @HassamSheikh, @chiayi, @petern48, @carolynwang, @MrKWatkins, @400Ping, @summaryzb, @peterxcli, @RocMarshal, @coqian, @yuhuan130, @ryankert01, @dayshah, @Anarion-zuo, @ZacAttack, @weimingdiit, @iamjustinhsu, @matthewdeng, @goutamvenkat-anyscale, @KeeProMise, @Sanskarzz, @yuchen-ecnu, @praneethkaturi, @rajeshg007, @ankur-anyscale, @Art0white, @xyuzh, @dancingactor, @MengjinYan, @dengkliu92, @alexeykudinkin

Ray Data

🔨 Fixes

• Disable hanging issue detection (#61895) — The hanging issue detector was making blocking calls to the Ray State API, which could cause the scheduling loop to block and severely degrade pipeline performance. The detector is disabled in this patch release until the blocking calls are fixed.

Ray Data

🎉 New Features

• Add checkpointing support to Ray Data (#59409)
• Compute Expressions: list operations (#59346), fixed-size arrays (#58741), string padding (#59552), logarithmic (#59549), trigonometric (#59712), arithmetic (#59678), and rounding (#59295)
• Add sql_params support to read_sql (#60030)
• Add AsList aggregation (#59920)
• Support CountDistinct aggregate (#59030)
• Add credential provider abstraction for Databricks UC datasource (#60457)
• Support callable classes for UDFExpr (#56725)
• Add autoscaler metrics to Data Dashboard (#60472)
• Add optional filesystem parameter to download expression (#60677)
• Allow specifying partitioning style or flavor in write_parquet() (#59102)
• New cluster autoscaler enabled by default (#60474)

💫 Enhancements

• Improve numerical stability in scalers by handling near-zero values (#60488)
• Export dataset operator output schema to event logger (#60086)
• Iceberg: add retry policy for Storage + Catalog writes (#60620)
• Iceberg: remove calls to Catalog Table in write tasks (#60476)
• Expose logical operators and rules via package exports (#60297, #60296)
• Demote Sort from requiring preserve_order (#60555)
• Improve appearance of repr(dataset) (#59631)
• Allow configuring DefaultClusterAutoscalerV2 thresholds via env vars (#60133)
• Use Arrow IPC for Arrow Schema serialization/deserialization (#60195)
• Store _source_paths in object store to prevent excessive spilling during read task serialization (#59999)
• Add more shuffle fusion rules (#59985)
• Enable and tune DownstreamCapacityBackpressurePolicy (#59753)
• Enable concurrency cap backpressure with tuning (#59392)
• Set default actor pool scale up threshold to 1.75 (#59512)
• Don't downscale actors if the operator hasn't received any inputs (#59883)
• Don't reserve GPU budget for non-GPU tasks (#59789)
• Only return selected data columns in hive-partitioned Parquet files (#60236)
• Ordered + FIFO bundle queue (#60228)
• Add node_id, pid, attempt number for hanging tasks (#59793)
• Revise resource allocator task scheduling to factor in pending task outputs (#60639)
• Track block serialization time (#60574)
• Use metrics from OpRuntimeMetrics for progress (#60304)
• Tabular form for streaming executor op metrics (#59774)
• Info-log cluster scale-up decisions (#60357)
• Use plain mode instead of grid mode for OpMetrics logging (#59907)
• Progress reporting refactors (#59350, #59629, #59880)
• Remove deprecated TENSOR_COLUMN_NAME constant (#60573)
• Remove meta_provider parameter (#60379)
• Decouple Ray Train from Ray Data by removing top-level ray.data imports (#60292)
• Move extension types to ray.data (#59420)
• Skip upscaling validation warning for fixed-size actor pools (#60569)
• Make StatefulShuffleAggregation.finalize allow incremental streaming (#59972)
• Revisit OutputSplitter semantics to avoid unnecessary buffer accumulation (#60237)
• Update to PyArrow 23 (#60739, #59489)
• Add BackpressurePolicy to streaming executor progress bar (#59637)
• Support Arrow-based transformations for preprocessors (#59810)
• StandardScaler preprocessor with Arrow format (#59906)
• OneHotEncoder with Arrow format (#59890)

🔨 Fixes

• Fuse MapBatches even if they modify the row count (#60756)
• Don't push limit past map_batches by default (#60448)
• Fix wrong type hint of other dataset in zip and union (#60653)
• Fix ActorPoolMapOperator to guarantee dispatch of all given inputs (#60763)
• Fix ArrowInvalid error when backfilling missing fields from map tasks (#60643)
• Fix attribute error in UnionOperator.clear_internal_output_queue (#60538)
• Fix DefaultClusterAutoscalerV2 raising KeyError: 'CPU' (#60208)
• Fix ReorderingBundleQueue handling of empty output sequences (#60470)
• Fix task completion time without backpressure grafana panel metric name (#60481)
• Fix Union operator blocking when preserve_order is set (#59922)
• Fix autoscaler requesting empty resources instead of previous allocation when not scaling up (#60321)
• Fix autoscaler not respecting user-configured resource limits (#60283)
• Fix DefaultAutoscalerV2 not scaling nodes from zero (#59896)
• Fix Iceberg warning message (#60044)
• Fix Parquet datasource path column support (#60046)
• Fix ProgressBar with use_ray_tqdm (#59996)
• Fix stale stats on refit for preprocessors (#60031)
• Fix StreamingRepartition hang with empty upstream results (#59848)
• Fix operator fusion bug to preserve UDF modifying row count (#59513)
• Fix AutoscalingCoordinator double-allocating resources for multiple datasets (#59740)
• Fix DownstreamCapacityBackpressurePolicy issues (#59990)
• Fix AutoscalingCoordinator crash when requesting 0 GPUs on CPU-only cluster (#59514)
• Fix TensorArray to Arrow tensor conversion (#59449)
• Fix resource allocator not respecting max resource requirement (#59412)
• Fix GPU autoscaling when max_actors is set (#59632)
• Fix checkpoint filter PyArrow zero-copy conversion error (#59839)
• Restore class aliases to fix deserialization of existing datasets (#59828, #59818)
• Fix DataContext deserialization issue with StatsActor (#59471)

📖 Documentation

• Sort references in "Loading data and Saving data" pages (#60084)
• Fix inconsistent heading levels in "How to write tests" guide (#60706)
• Clarify resource_limits refers to logical resources (#60109)
• Update read_lance doc (#59673)
• Fix broken link in read_unity_catalog docstring (#59745)
• Fix bug in docs for enable_true_multi_threading (#60515)
• Add more education around transformations (#59415)

Ray Serve

🎉 New Features

• Queue-based autoscaling for TaskConsumer deployments (phase 1). Introduces a QueueMonitor actor that queries message brokers (Redis, RabbitMQ) for queue length, enabling TaskConsumer scaling based on pending tasks rather than HTTP load. (#59430)
• Default autoscaling parameters for custom policies. New apply_autoscaling_config decorator allows custom autoscaling policies to automatically benefit from Ray Serve's standard parameters (delays, scaling factors, bounds) without reimplementation. (#58857)
• label_selector and bundle_label_selector in Serve deployments. Deployments can now specify node label selectors for scheduling and bundle-level label selectors for placement groups, useful for targeting specific hardware (e.g., TPU topologies). (#57694)
• Deployment-level autoscaling observability. The controller now emits a structured JSON serve_autoscaling_snapshot log per autoscaling-enabled deployment each control-loop tick, with an event summarizer that reduces duplicate logs. (#56225)
• Batching with multiplexing support. Batching now guarantees each batch contains requests for the same multiplexed model, enabling correct multiplexed model serving with @serve.batch. (#59334)

💫 Enhancements

• Replica routing data structure optimizations. O(1) pending-request lookups, cached replica lists, lazy cleanup, optimized retry insertion, and metrics throttling yield significant routing performance improvements. (#60139)
• New operational metrics suite. Added long-poll metrics, replica lifecycle metrics, app/deployment status metrics, proxy health and request routing delay metrics, event loop utilization metrics, and controller health metrics — greatly improving monitoring and debugging capabilities. (#59246, #59235, #59244, #59238, #59535, #60473)
• Autoscaling config validation. lookback_period_s must now be greater than metrics_interval_s, preventing silent misconfigurations. (#59456)
• Cross-version root_path support for uvicorn. root_path now works correctly across all uvicorn versions, including >=0.26.0 which changed how root_path is processed. (#57555)
• Preserve user-set gRPC status codes. When deployments raise exceptions after setting a gRPC status code on the context, that code is now correctly propagated to the client instead of being overwritten with INTERNAL. Error messages are truncated to 4 KB to respect HTTP/2 trailer limits. (#60482)
• Replica ThreadPoolExecutor capped to num_cpus. The user-code event loop's default ThreadPoolExecutor is now limited to the deployment's num_cpus, preventing oversubscription when using asyncio.to_thread. (#60271)
• Generic actor registration API for shutdown cleanup. Deployments can register auxiliary actors (e.g., PrefixTreeActor) with the controller for automatic cleanup on serve.shutdown(), eliminating cross-library import dependencies. (#60067)
• Deployment config logging in controller. Deployment configurations are now logged in the controller for easier debugging and auditability. (#59222, #59501)
• Pydantic v1 deprecation warning. A FutureWarning is now emitted at ray.init() when Pydantic v1 is detected, as support will be removed in Ray 2.56. (#59703)

🔨 Fixes

• Fixed tracing signature mismatch across processes. Resolved TypeError: got an unexpected keyword argument _ray_trace_ctx when calling actors from a different process than the one that created them (e.g., serve start + dashboard interaction). (#59634)
• Fixed ingress deployment name collision. Ingress deployment name was incorrectly modified when a child deployment shared the same name, causing routing failures. (#59577)
• Fixed downstream deployment over-provisioning. Downstream deployments no longer over-provision replicas when receiving DeploymentResponse objects. (#60747)
• Fixed replicas hanging forever during draining. Replicas no longer hang indefinitely when requests are stuck during the draining phase. (#60788)
• Fixed TaskProcessorAdapter shutdown during rolling updates. Removed shutdown() from __del__, which was broadcasting a kill signal to all Celery workers instead of just the local one, breaking rolling updates. (#59713)
• Fixed Windows test failures. Resolved tracing file handle cleanup on Windows, skipped incompatible gRPC and tracing tests on Windows. (#60078, #60356, #60393, #59771)
• Fixed flaky tests. Addressed gauge throttling race in test_router_queue_len_metric, ensured proxy replica queue cache is populated before GCS failure tests, and added metrics server readiness checks. (#60333, #60466, #60468)
• Fixed distilbert test segfault. Worked around a pyarrow/jemalloc crash triggered by specific import ordering of FastAPI, torch, and TensorFlow. (#60478)

📖 Documentation

• Improved autoscaling documentation. Clarified the relationship between delays, metric push intervals, and the autoscaling control loop. (#59475)
• New example: video analysis inference. End-to-end notebook demonstrating a Serve application for scene change detection, - tagging, and video description. (#59859)
• New examples: model multiplexing and model composition. Published workload-based examples for forecasting with model multiplexing and recommendation systems with model composition. (#59166)
• Model registry integration guide. Added documentation for integrating Serve with model registries (e.g., MLflow). (#59080)
• Fixed broken documentation links. Resolved 404 errors for async inference, MLflow registry example, and LLM code examples. (#59917, #60071, #59520, #59521, #60181)
• Fixed monitoring docs. Corrected target replicas metric emission to enable time-series comparison with actual replicas. (#59571)
• Async inference template. Added an end-to-end template for building asynchronous inference applications with Ray Serve. (#58393, #59926)

🏗 Architecture refactoring

• Environment variable cleanup (5-part series). Removed deprecated and redundant env vars (RAY_SERVE_DEFAULT_HTTP_HOST, RAY_SERVE_DEFAULT_HTTP_PORT, RAY_SERVE_DEFAULT_GRPC_PORT, RAY_SERVE_HTTP_KEEP_ALIVE_TIMEOUT_S, RAY_SERVE_REQUEST_PROCESSING_TIMEOUT_S, RAY_SERVE_ENABLE_JSON_LOGGING, RAY_SERVE_ALWAYS_RUN_PROXY_ON_HEAD_NODE), cleaned up legacy constant fallbacks, and added documentation for previously undocumented env vars (e.g., RAY_SERVE_CONTROLLER_MAX_CONCURRENCY, RAY_SERVE_ROOT_URL, proxy health check settings, and fault tolerance params). Users relying on removed env vars should migrate to the Serve config API (http_options, grpc_options, LoggingConfig). (#59470, #59619, #59647, #59963, #60093)

Ray Train

🎉 New Features

• Add TPU multi-slice support to JaxTrainer (#58629)
• Update async validation API (#59428)
• Add a CallbackManager and guardrail some callback hooks (#60117)
• Add inter-execution file shuffling for deterministic multi-epoch training (#59528)
• Resume validations on driver restoration (#59270)

💫 Enhancements

• Pass ray remote args to validation task (#60203)
• Deprecate Predictor API (#60305)
• Increase worker group start default timeout to 60s (#60376)
• Unify PlacementGroup and SlicePlacementGroup interface in WorkerGroup (#60116)
• Cleanup zombie RayTrainWorker actors (#59872)
• Add usage telemetry for checkpointing and validation (#59490)
• Validate that validation is called with a checkpoint (#60548)
• Replace pg.ready() with pg.wait() in worker group (#60568)
• Rename DatasetsSetupCallback to DatasetsCallback (#59423)
• Update "Checkpoint Report Time" metric title to "Cumulative Checkpoint Report Time" (#58470)
• Add training failed error back to failure policy log (#59957)
• Decouple Ray Train from Ray Data by removing top-level imports (#60292)

🔨 Fixes

• Add try-except for pg.wait() (#60743)
• TrainController reraises AsyncioActorExit (#59461)

📖 Documentation

• Add a JaxTrainer template (#59842)
• Update Jax doc to include GPU and multi-slice TPU support (#60593)
• Document checkpoint_upload_fn backend and cuda:nccl backend support (#60541)
• Rename checkpoint_upload_func to checkpoint_upload_fn in docs (#60390)
• Fix Ray Train workloads and PyTorch with ASHA templates (#60537)
• Publish Ray Train workload example (#58936)

Ray Tune

🔨 Fixes

• Avoid file deletion race by using unique tmp file names (#60556)

Ray LLM

🎉 New Features

• Add /tokenize and /detokenize endpoints (#59787)
• Add /collective_rpc endpoint for RLHF weight synchronization (#59529)
• Add Control Plane API for Sleep/Wakeup (#59455)
• Add Pause/Resume Control Plane API (#59523)
• Add support for classification and scoring models (#59499)
• Add pooling parameter (#59534)
• Support vLLM structured outputs with backward-compat for guided_decoding (#59421)
• Add CPU support to Ray Serve LLM (#58334)
• Add should_continue_on_error support for ServeDeploymentStage (#59395)
• Support configuring HttpRequestUDF resources (#60313)

💫 Enhancements

• Upgrade vLLM to 0.15.0 (#60679)
• Unify schema of success and failure rows (#60572)
• Prefer uniproc executor over mp executor when world_size==1 (#60403)
• Use compute instead of concurrency to specify ActorPool size (#59645)
• Remove DataContext overrides in Ray Data LLM Processor (#60142)
• Use numpy arrays for embeddings to avoid torch.Tensor serialization overhead (#59919)
• Make PrefixCacheAwareRouter imbalance threshold less surprising (#59390)
• Allow tokenized_prompt without prompt in vLLMEngineStage (#59801)
• Avoid passing enums through fn_constructor_kwargs (#59806)
• Refactor Control Plane endpoints into mixins (#59502)
• Remove CUDA_VISIBLE_DEVICES deletion workaround (#60502)

🔨 Fixes

• Fix nested dict to Namespace conversion in vLLM engine initialization (#60380)
• Fix JSON non-serializable ndarray exception in http_request_stage (#60299)
• Exit actor on EngineDeadError to enable recovery (#60145)
• Fix NIXL port conflict in prefill-decode disaggregation test (#60057)

📖 Documentation

• Batch inference docs reorg and update to reflect per-stage config refactor (#59214)
• Add resiliency section and refine doc code (#60594)
• Add video/audio examples for vLLMEngineProcessor (#59446)
• Add SGLang integration example (#58366)
• Remove inaccurate statement in docs (#60425)

Ray RLlib

🎉 New Features

• Add TQC (Truncated Quantile Critics) algorithm implementation (#59808)
• Add LR scheduling ability to BC and MARWIL (#59067)
• RLlib and Ray Tune: Hyperparameter Optimisation example (#60182)

💫 Enhancements

• 🔥 APPO improvements: learner pipeline performance improvements (#59544)
• Improve stateful model training on offline data (#59345)
• Create resource bundle per learner (#59620)
• Improve env runner sampling by replacing recursive solution with iterative solution (#56082)
• Improve IMPALA examples and premerge (#59927)
• Remove MLAgents dependency (#59524)
• Upgrade to gymnasium v1.2.2 (#59530)
• Decrease log quantity for learning tests (#59005)
• Update learner state warnings to the debug level (#60178)
• Don't log np.nanmean warnings in EMA stats (#60408)

🔨 Fixes

• Fix DQN RLModule forward methods to handle dict spaces (#60451)
• Fix LearnerGroup.load_module_state() and mark as deprecated (#60354)
• Fix static dimension issue in ONNX export of Torch attention models (#60102)
• Fix Multi-Agent Episode concatenation for sequential environments (#59895)
• Fix module episode returns metrics accumulation for shared module IDs (#60234)
• Fix rollout fragment length calculation in AlgorithmConfig (#59438)
• Fix checkpointable issues with cloud storages (#60440)
• Update flatten_observations.py for nested spaces for ignored multi-agent (#59928)

Ray Core

🎉 New Features

• Resource Isolation: unify config construction, add public docs, and expose cgroup_path in ray.init() (#59372, #60183, #60726)
• Support tensor-level deduplication for NIXL (#60509)
• Add CUDA IPC transport for RDT (#59838)
• Register custom transport at runtime for RDT (#59255)
• Support TPU v7x accelerator type for device discovery (#60338)
• Introduce local port service discovery (#59613)
• Cancel sync actor by checking is_canceled() (#58914)
• Support labels for ray job submit --entrypoint-resource (#59735)
• Add --ip option in ray attach (#59931)
• Add bearer token support for remote URI downloads (#60050)
• Support HTTP redirection download (#59384)
• Add ray kill-actor --name/--namespace for force/graceful shutdown (#60258)

💫 Enhancements

• Bound object spilling file size to avoid disk increase pressure (#60098)
• Replace SHA-1 with SHA-256 for internal hash operations (#60242)
• Use whitelist approach to block mutation requests from browser (#60526)
• Pass authentication headers to WebSocket connections in tail_job_logs (#60346)
• Add auth to Dashboard HTTP agent and client (#59891)
• Use dedicated service account path for Ray auth tokens (#60409)
• Update Kubernetes token auth verb to ray:write (#60411)
• Replace RAY_AUTH_MODE=k8s with separate config for Kubernetes token auth (#59621)
• Optimize token auth: use shared_ptr caching and avoid per-RPC construction (#59500)
• Optimize OpenTelemetry metric recording calls (#59337)
• Throttle infeasible resource warning (#59790)
• Add default excludes for working_dir uploads (#59566)
• Tell users why objects cannot be reconstructed (#59625)
• Extend instance allocation timeout in autoscaler v2 (#60392)
• Remove GCS centralized scheduling (#59979, #60121, #60188)
• Demote stale sync message drop log to DEBUG in RaySyncer (#59616)
• Migrate remaining std::unordered_map to absl::flat_hash_map (#59921)
• Add missing fields to NodeDefinitionEvent proto (#60314)
• Add actor and task event missing fields (#60287)
• Add node id to the base event (#59242)
• Add repr_name to actor_lifecycle_event (#59925)
• Support ALL in exposable event config (#59878)
• Support publishing events from aggregator to GCS (#55781)
• Update the attempt number of actor creation task when actor restarts (#58877)
• Unify node feasibility and availability checking for GPU fractions (#59278)
• Update TPU utils for multi-slice compatibility (#59136)
• Improve SubprocessModuleHandle.destroy_module() resource cleanup (#60172)
• Support viewing PIDs for Dashboard and Runtime Env Agent (#58701)
• Optimize autoscaler monitor by moving resource demand parsing outside loop (#59190)
• Avoid GCS query for is_head in dashboard agent startup (#59378)
• Skip reporter and event aggregator client creation in minimal mode (#59846)
• Support out-of-order actors by extracting metadata when creating (RDT) (#59610)
• Synchronize CUDA stream before registering for NIXL (#60072)
• Atomically send/recv for two-sided ordering (RDT) (#60202)
• Add get_session_name() to RuntimeContext (#59469)
• Make MAX_APPLICATION_ERROR_LEN configurable via env var (#59543)
• Preserve function signatures through Ray decorators (#60479)

🔨 Fixes

• Fix idle_time_ms resetting for nodes not running tasks (#60581)
• Fix task event loss during shutdown (#60247)
• Filter bad subscriber messages from taking down GCS publisher (#60252)
• Fix RAY_EXPERIMENTAL_NOSET_* environment variable parsing in accelerator managers (#60577)
• Fix ray start --no-redirect-output crash (#60394)
• Fix drain state propagation race condition (#59536)
• Fix use-after-free race condition in OpenTelemetry gauge metric callback during shutdown (#60048)
• Fix PSUTIL_PROCESS_ATTRS returning empty list on Windows (#60173)
• Fix deadlock in garbage collection when holding lock (#60014)
• Fix incorrect error handling in autoscaler for available_node_types on on-prem clusters (#60184)
• Fix invalid status transitions in autoscaler v2 (#60412, #59550)
• Fix GCS crash from race condition in MetricsAgentClient exporter initialization (#59611)
• Fix tracing signature mismatch when calling actors from different processes (#59634)
• Fix crash when killing actor handle from previous session (#59425)
• Fix multiple deployment same name resolve (#59577)
• Handle dual task errors with read-only args (#59507)
• Handle exceptions raised by internal_ip() within StandardAutoscaler (#57279)
• Fix uv_runtime_env_hook.py to pin worker Python version (#59768)
• Fix STRICT_PACK placement groups ignoring bundle label selectors (#60170)
• Fix logging bug when log value is an empty string (#59434)
• Fix aggregator-to-GCS event conversion (#59783)
• Raise error on tail log job error in newer Ray versions (#59506)
• Fix num retries left message (#59829)
• Fix psutil internal API usage in dashboard disk usage reporting (#59659)
• Fix event exporter init ray check (#60073)
• Prevent use-after-free error in core worker shutdown (#58435)
• Fix task name inconsistency in RUNNING vs FINISHED metrics (#59893)
• Fix symmetric_run using wrong condition to check GCS readiness (#59794)
• Preserve Pydantic details when serialization fails (#59401)
• Retry GCP project metadata updates on HTTP 412 errors (#60429)
• Fix v1 autoscaler TypeError when using bundle_label_selectors (#59850)
• Shorten SHA-256 hex with base32 to comply with GCP label limits (#60722)

📖 Documentation

• Add initial user guide for Ray resource isolation with writable cgroups (#59051)
• Add token authentication internals documentation (#59299)
• Update metric exporter docs (#59874)
• Add internal documentation for Port Service Discovery (#59844)
• Update misleading Ray job diagram (#59940)
• Add debugging logs related to pinned argument size limit (#60175)
• Add slow startup tip to podman troubleshooting docs (#59942)
• Clarify ray.shutdown() behavior for local vs remote clusters (#59845)
• Improve placement group fault tolerance doc (#59830)
• Add head-node memory growth and OOM guidance (#58695)
• Add documentation for RAY_RUNTIME_ENV_BEARER_TOKEN env var (#60136)

Dashboard

💫 Enhancements

• Support more panels in dashboard (#60018)
• Add autoscaler metrics to Data Dashboard (#60472)
• Support viewing PIDs for Dashboard and Runtime Env Agent (#58701)

🔨 Fixes

• Update total for dark mode color (#60106)

Ray Wheels and Images

• Add experimental Python 3.13 images (#59811, #60549)
• Drop Python 3.9 support completely (#59824, #59825, #59769); wheel Python version now declared as 3.10+ (#59835)
• ray-cpp wheels are now py3-none, without specific Python versions. (#59969)

Documentation

• Add committership documentation (#60069)
• Update contribution guide with common labels (#59473)
• Add KubeRay & Volcano integration docs update (#59636)
• Add RayJob InTreeAutoscaling with Kueue docs after Kueue 0.16.0 release (#59648)
• Refactor LLM batch inference template (#59897)
• Add async inference template (#58393)
• Add RunLLM chat widget for Ray docs (#59126)
• Fix various typos and broken links (#60249, #59901, #60181)
• Replace Ray Tune + Train example with vanilla Ray Tune in homepage (#60229)
• Add Ray technical charter (#60068)

Thanks

Thank you to everyone who contributed to this release! @KaisennHu, @MiXaiLL76, @slfan1989, @krisselberg, @JasonLi1909, @Priya-753, @pseudo-rnd-thoughts, @zzchun, @ZacAttack, @pushpavanthar, @jjyao, @ryanaoleary, @pcmoritz, @akshay-anyscale, @HassamSheikh, @yurekami, @Hyunoh-Yeo, @ruoliu2, @nrghosh, @wxwmd, @myandpr, @J-Meyers, @trilamsr, @kouroshHakha, @limarkdcunha, @manhld0206, @jreiml, @preneond, @yuchen-ecnu, @Yicheng-Lu-llll, @AchimGaedkeLynker, @vaishdho1, @israbbani, @OneSizeFitsQuorum, @Sathyanarayanaa-T, @nadongjun, @xinyuangui2, @Rob12312368, @as-jding, @lee1258561, @popojk, @coqian, @rajeshg007, @jeffreywang-anyscale, @kamil-kaczmarek, @alexeykudinkin, @Aydin-ab, @mgchoi239, @dragongu, @edoakes, @smortime, @tk42, @abrarsheikh, @jakubzimny, @Future-Outlier, @axreldable, @owenowenisme, @g199209, @cem-anyscale, @dayshah, @akelloway, @daiping8, @dlwh, @robertnishihara, @400Ping, @matthewdeng, @antoine-galataud, @cristianjd, @Partth101, @goutamvenkat-anyscale, @codope, @seanlaii, @andrew-anyscale, @andrewsykim, @liulehui, @simonsays1980, @Sparks0219, @yifanmai, @landscapepainter, @win5923, @kangwangamd, @srinarayan-srikanthan, @KeeProMise, @srinathk10, @my-vegetable-has-exploded, @MengjinYan, @yancanmao, @yuhuan130, @ArturNiederfahrenhorst, @akyang-anyscale, @rushikeshadhav, @kongjy, @harshit-anyscale, @justinvyu, @dancingactor, @Vito-Yang, @cr7258, @marwan116, @muyihao, @DeborahOlaboye, @bveeramani, @kriyanshii, @khluu, @machichima, @Kunchd, @jonded94, @iamjustinhsu, @sampan-s-nayak, @wingkitlee0, @sunsetxh, @dkhachyan, @can-anyscale, @TimothySeah, @raulchen, @elliot-barn, @ryankert01, @xyuzh, @stephanie-wang, @hao-aaron, @simeetnayan81, @cszhu, @richardliaw, @yuanjiewei, @kyuds, @eicherseiji, @RedGrey1993, @rueian, @jeffreyjeffreywang, @crypdick, @ankur-anyscale, @aslonnie

Highlights

• Ray plans to drop support for Pydantic V1 starting version 2.56.0. Please see this RFC for details.
• Ray Data now has support for bounded reading from Kafka and improved Iceberg support.

Ray Data

🎉 New Features

• Autoscaling: New utilization-based cluster autoscaler for Ray Data workloads (#59353, #59362, #59366). To use this new autoscaler set RAY_DATA_CLUSTER_AUTOSCALER=V2.
• Kafka Datasource: Add Kafka as a native datasource for data ingestion (#58592)
• Dataset summary API: Add Dataset.summary() API for quick dataset inspection (#58862)
• Iceberg support: Add Iceberg schema evolution, upsert, and overwrite support (#59210, #59335)
• Graceful error handling: Add should_continue_on_error for graceful error handling in batch inference (#59212)
• Datetime compute expressions: Add datetime compute expressions support (#58740)
• Grouped with_column expressions: Enable expressions for grouped with_column in Ray Data (#58231)
• Parallelized collation: Parallelize DefaultCollateFn, arrow_batch_to_tensors (#58821)

💫 Enhancements

• Optimized Autoscaler Step Size: Optimize autoscaler to support configurable step size for actor pool scaling (#58726)
• Improved Streaming Repartition: Improve streaming repartition performance (#58728)
• Actor init retry: Add actor retry if there's a failure in __init__ (#59105)
• Fused Repartition + MapBatches: Fuse StreamingRepartition with MapBatches operators to scale collate (#59108)
• Combined repartitions: Combine consecutive repartitions for efficiency (#59145)
• Prefetch buffering: Handle prefetch buffering in iter_batches (#58657)
• HashShuffle block breakdown: HashShuffleAggregator breaks down blocks on finalize (#58603)
• Backpressure tuning: Tune concurrency cap backpressure object store budget ratio (#58813)
• Non-string ApproximateTopK: Support non-string items for ApproximateTopK aggregator (#58659)
• Lance version support: Add version support to read_lance() (#58895)
• Dashboard metrics: Add time_to_first_batch and get_ref_bundles metrics to data dashboard (#58912)
• Iter prefetched bytes stats: Add iter_prefetched_bytes statistics tracking (#58900)
• Configurable batching for iter_batches: Add configurable batching for resolve_block_refs to speed up iter_batches (#58467)
• Improved dashboard metrics: Improve Ray Data dashboard metrics display (#58667)
• Histogram percentiles: Update Ray Data histograms to show percentiles in data dashboard (#58650)
• Deprecated API removal: Remove deprecated read_parquet_bulk API (#58970)
• Block shaping option: Add disable block shaping option to BlockOutputBuffer (#58757)
• Removed concurrency lock: Remove concurrency lock for better performance (#56798)

🔨 Fixes

• Fixes to Unique: Fix support of list types for Unique aggregator (#58916)
• Parquet NaN fix: Fix reading from written parquet for numpy with NaNs (#59172)
• Hash Shuffle empty block: Fix empty block sort in hash shuffle operator (#58836)
• Hive partitioning pushdown: Fix pushdown optimizations with Hive partitioning (#58723)
• Object Store usage reporting: Fix obj_store_mem_max_pending_output_per_task reporting (#58864)
• Pyarrow FileSystem serialization fix: Handle filesystem serialization issue in get_parquet_dataset (#57047)
• Azure UC SAS: Handle Azure UC user delegation SAS (#59393)
• Async UDF Thread Cleanup: Close threads from async UDF after actor died (#59261)
• Object Locality Default: Default return 0s for object locality instead of -1s (#58754)

📖 Documentation

• Added contributing guide to Ray Data documentation (#58589)
• Added download expression to key user journeys in documentation (#59417)
• Added Kafka user guide (#58881)
• Added unstructured data templates from Ray Summit 2025 (#57063)
• Improved instructions for reading Hugging Face datasets (#58492, #58832)
• Refined batch-format guidance in docs (#58971)
• Exposed vision_preprocess and vision_postprocess in VLM docs (#59012)
• Added upgrading huggingface_hub instruction (#59109)
• Added scaling out expensive collation functions doc (#58993)

Ray Serve

🎉 New Features

• Deployment topology visibility. Exposes deployment dependency graphs in Serve REST API, allowing users to visualize and understand the DAG structure of their applications. (#58355)
• External autoscaler integration. Adds external_scaler_enabled flag to application config, enabling third-party autoscalers to control replica counts. (#57727, #57698)
• Node rank and local rank support. Extends replica rank system to track node-level and per-node local ranks, enabling better distributed serving coordination for multi-node deployments. (#58477, #58479)
• Custom batch size function. Allows users to define custom functions for computing logical batch sizes in @serve.batch, useful when batch items have varying weights (e.g., token counts in LLM inference). (#59059)
• Stateful application-level autoscaling. Adds policy state persistence for custom autoscaling policies, allowing policies to maintain state across control-loop iterations. (#59118)
• New autoscaling, batching, and routing metrics. Adds Prometheus metrics for autoscaling decisions (ray_serve_deployment_target_replicas, ray_serve_autoscaling_decision_replicas), batching statistics, and router queue latency for improved observability. (#59220, #59232, #59233)

💫 Enhancements

• Smarter downscaling behavior. Prioritizes stopping most recently scaled-up replicas during downscale, preserving long-lived replicas that are optimally placed and fully warmed up. (#52929)
• Autoscaling performance optimizations. Short-circuits metric aggregation for single time series cases (O(n log n) → O(1)) and lazily evaluates expensive autoscaling context fields to reduce controller CPU usage. (#58962, #58963)
• Route matching cleanup. Removes redundant route matching logic from replicas since correct route values are now included in RequestMetadata. Also allows multiple methods (GET, PUT) corresponding to a route. (#58927)
• Deployment wrapper metadata preservation. Wrapper classes from decorators like @ingress now preserve original class metadata (__qualname__, __module__, __doc__, __annotations__). (#58478)
• Improved type annotations. Enhances generic type annotations on DeploymentHandle, DeploymentResponse, and DeploymentResponseGenerator for better IDE support and type inference. Adds .result() stub to DeploymentResponseGenerator to fix static typing errors. (#59363, #58522)

🔨 Fixes

• YAML serialization for autoscaling enums. Fixes RepresenterError when using serve build with AggregationFunction enum values in autoscaling config. (#58509)
• Autoscaling context timestamp fix. Correctly sets last_scale_up_time and last_scale_down_time on autoscaling context. (#59057)
• Deadlock in chained deployment responses. Fixes hang when awaiting intermediate DeploymentResponse objects in a chain of deployment calls from different event loops. (#59385)
• FastAPI class-based view inheritance. Fixes make_fastapi_class_based_view to properly handle inherited methods. (#59410)

📖 Documentation

• Async I/O best practices guide. New documentation covering async programming patterns and best practices for Ray Serve deployments. (#58909)
• Replica scheduling guide. New documentation covering compact scheduling, placement groups, custom resources, and guidance on when to use each feature. (#59114)

Ray Train

🎉 New Features

• Worker Placement with Label Selectors: Added label_selector to ScalingConfig. This allows users to control worker placement by targeting specific labeled nodes in the cluster. (#58845, #59414)
• Multihost JaxTrainer on GPU: Introduced support for JaxTrainer running on GPU machines. (#58322)
• Checkpoint Consistency Modes: Added CheckpointConsistencyMode to get_all_reported_checkpoints, providing options for handling checkpoint retrieval consistency. (#58271)
• Per-Dataset Execution Options: DataConfig now supports setting execution_options on a per-dataset basis for finer-grained control over data loading. (#58717)

💫 Enhancements

• Nested Metrics Support: Result.get_best_checkpoint now supports nested metrics, allowing for more flexible metric tracking and checkpoint selection. (#58537)
• Non-Blocking Checkpoint Retrieval: get_all_reported_checkpoints no longer blocks when only metrics are reported. (#58870)
• Improved Resource Cleanup: Implemented eager cleanup of data resources and placement groups upon training run failures or aborts, preventing resource leaks. (#58325, #58515)

🔨 Fixes

• MLflow Compatibility: Updated setup_mlflow API to ensure full compatibility with Ray Train V2. (#58705)
• Validation for Checkpoint Uploads: A ValueError is now raised if checkpoint_upload_fn fails to return a valid checkpoint. (#58863)

📖 Documentation

• New API Documentation: Added comprehensive documentation for the ray.train.get_all_reported_checkpoints method. (#58946)

Ray Tune

💫 Enhancements:

• Nested Metrics Support: Result.get_best_checkpoint now supports nested metrics, allowing for more flexible metric tracking and checkpoint selection. (#58537)

Ray LLM

💫 Enhancements

• Cloud filesystem restructuring with provider-specific implementations (#58469)
• Bump transformers to 4.57.3 (#58980)
• Ray Data LLM config refactor (#58298)
• Update vllm_engine.py to check for VLLM_USE_V1 attribute (#58820)
• Infer VLLM_RAY_PER_WORKER_GPUS from fractional placement-group bundles automatically (#58949)

🔨 Fixes

• Fix LLM DP release test configuration (#59090)

Ray RLlib

🎉 New Features

• DreamerV3: allow num_env_runners \> 0 (#58495)

💫 Enhancements

• 🔥 MetricsLogger tweaks+ Stats rewrite (#56838)
• move restart message into EnvRunner (#56750)
• make “Footsies” less verbose (optionally) (#58939)
• update an AlgorithmConfig deprecated argument with incorrect behavior/semantics (#59138)
• Examples/docs cleanup:
  • merge tuned examples into examples/ (#58893)
  • move old API examples (#59159)
  • move example run scripts (#59160)
  • remove Torch 2.x doc tied to removed benchmarks (#59173)
  • remove rllib/benchmark(s) folder from RLlib directory (#59158)
• Testing / CI & infra cleanup (part of a larger effort to organize + harden RLlib testing):
  • clean up tests folder layout in favor of /component/tests (#58890)
  • re-enable and fix nightly tests for APPO on Atari and MuJoCo (#58853)
  • re-enable all RLlib doctests (#58974)
  • add pytest reporting hook (pytest_runtest_makereport) across tests (#59003)
  • add/enable RLlib Py3.10 CI lane (#59226)
  • fix as-release-test silently failing (#59386)
  • fix recursive imports in old test-utils location (#59435)
  • Remove asv.conf.json (#58934)
  • Update requirement for byod_rllib.sh (#59157)

🔨 Fixes

• Fix custom model-config mismatch between EnvRunner and Learner (#58739)
• MultiAgentEnvRunner: prevent double-calling connectors (#58931)
• Error handling: log or raise when a case is not fully handled (#58889)
• Error handling: error out when data cannot be loaded (#59002)
• Assorted RLlib bugfixes (#59386)

📖 Documentation

• Update APPO paper reference to link to IMPACT paper (#58935)

Ray Core

🎉 New Features

• Support zero-copy serialization for read-only PyTorch tensors via RAY_ENABLE_ZERO_COPY_TORCH_TENSORS (#57639)
• Add .rayignore file support for controlling cluster uploads (#58500)
• Improve large-scale resource view synchronization through sync message batching (#57641)
• Autoscaler with cloud resource availability awareness (#58623)
• Token authentication UX improvements with new AuthenticationError exception (#58737)
• Support X-Ray-Authorization fallback header for auth token in dashboard (#58819)

💫 Enhancements

• Limit core worker gRPC reply threads to 2 by default via RAY_core_worker_num_server_call_thread (#58771)
• Make accessor node address and liveliness cache thread safe (#58947)
• Create OtlpGrpcMetricExporter wrapper to log export failures (#58929)
• Print detailed exception information when failing to report events (#58953)
• Simplify local/global GC logic (#58671)
• Surface correct error message when get_if_exists=True for actor lookup (#58628)
• Throw AuthenticationError from Python for token loading errors (#59031)
• Use secrets.token_hex(32) to generate auth tokens (#58818)
• Remove AUTH_MODE=token check in get-auth-token CLI (#58848)
• Introduce core chaos network release tests (#58868)

🔨 Fixes

• Fix grpc_authentication_server_interceptors streaming response handling (#59104)
• Fix handle leak in IsProcessAlive on Windows (#59106)
• Fix counter metric default branch for RAY_enable_open_telemetry (#59095)
• Fix leaking metric recorder in tests (#58952)
• Fix crash when using JVM HDFS by adding RAY_DISABLE_FAILURE_SIGNAL_HANDLER option (#58984)
• Fix heap corruption in RayletClient causing driver crash (use-after-free) (#58660)
• Use shared_ptr for pins_in_flight_ to prevent use-after-free (#58744)
• Remove deprecated add_command_alias (#58719)
• Remove cluster_full_of_actors_detected_* fields (unused in autoscaler v2) (#59052)

📖 Documentation

• Add token-auth.md documentation page (#58829)
• Update KubeRay authentication guide to use native Ray token authentication (#58729)

Dashboard

💫 Enhancements

• Add time_to_first_batch and get_ref_bundles metrics to data dashboard (#58912)
• Update Ray Data histograms to show percentiles grouped by operator (#58650)

Ray Wheels and Images

• Upgraded rich, cupy-cuda12x, and memray (#58983)
• Upgraded lxml to 6.0.2 (#58808)
• Upgraded requests from 2.32.3 to 2.32.5 (#58724)
• Added openlineage-python in the dependency set (#58724)

Thanks

Thank you to everyone who contributed to this release! @xinyuangui2, @harshit-anyscale, @Sparks0219, @israbbani, @siyuanfoundation, @robertnishihara, @thomasdesr, @spencer-p, @aslonnie, @ZacAttack, @soodoshll, @marosset, @simeetnayan81, @soffer-anyscale, @abrarsheikh, @400Ping, @richo-anyscale, @as-jding, @rueian, @kshanmol, @yancanmao, @zzchun, @coqian, @matthewdeng, @Future-Outlier, @YoussefEssDS, @ykdojo, @pseudo-rnd-thoughts, @lowdy1, @ArturNiederfahrenhorst, @myandpr, @komikndr, @machichima, @RisinT96, @curiosity-hyf, @alanwguo, @CaiZhanqi, @Aydin-ab, @MengjinYan, @suzuri-lollipop, @jeffreyjeffreywang, @rushikeshadhav, @alexeykudinkin, @meAmitPatil, @zcin, @teddygood, @elliot-barn, @dayshah, @srinathk10, @XLC127, @simonsays1980, @kevin85421, @bveeramani, @kunling-anyscale, @khluu, @andrew-anyscale, @KaisennHu, @kouroshHakha, @ryankert01, @pavitrabhalla, @jjyao, @dragongu, @SolitaryThinker, @justinrmiller, @wxwmd, @Haustle-v, @TimothySeah, @goutamvenkat-anyscale, @liulehui, @raulchen, @HassamSheikh, @Priya-753, @vaishdho1, @dancingactor, @daiping8, @eloaf, @JasonLi1909, @rayci-bot, @richardliaw, @SheldonTsen, @Yicheng-Lu-llll, @ktyxx, @pschmutz, @iamjustinhsu, @ahao-anyscale, @cem-anyscale, @eicherseiji, @edoakes, @rajeshg007, @arki05, @andrewsykim, @nrghosh, @ryanaoleary, @kyuds, @Daraan, @can-anyscale, @sampan-s-nayak, @xyuzh, @owenowenisme

• Fix for CVE-2025-62593: reject Sec-Fetch-* other browser-specific headers in dashboard browser rejection logic

• More robust handling for CVE-2025-62593: test for more browser-specific headers in dashboard browser rejection logic

Release Highlights

Ray Core:

• End of Life for Python 3.9 Support: Ray will no longer be releasing Python 3.9 wheels from now on.
• Token authentication: Ray now supports built-in token authentication across all components including the dashboard, CLI, API clients, and internal services. This provides an additional layer of security for production deployments to reduce the risk of unauthorized code execution. Token authentication is initially off by default. For more information, see: https://docs.ray.io/en/latest/ray-security/token-auth.html

Ray Data:

• We’ve added a number of improvements for Iceberg, including upserts, predicate and projection pushdown, and overwrite.
• We’ve added significant improvements to our expressions framework, including temporal, list, tensor, and struct datatype expressions.

Ray Libraries

Ray Data

🎉 New Features:

• Added predicate pushdown rule that pushes filter predicates past eligible operators (#58150, #58555)
• Iceberg support for upsert tables, schema updates, and overwrite operations (#58270)
• Iceberg support for predicate and projection pushdown (#58286)
• Iceberg write datafiles in write() then commit (#58601)
• Enhanced Unity Catalog integration (#57954)
• Namespaced expressions that expose PyArrow functions (#58465)
• Added version argument to read_delta_lake (#54976)
• Generator UDF support for map_groups (#58039)
• ApproximateTopK aggregator (#57950)
• Serialization framework for preprocessors (#58321)
• Support for temporal, list, tensor, and struct datatypes (#58225)

💫 Enhancements:

• Use approximate quantile for RobustScaler preprocessor (#58371)
• Map batches support for limit pushdown (#57880)
• Make all map operations zero-copy by default (#58285)
• Use tqdm_ray for progress reporting from workers (#58277)
• Improved concurrency cap backpressure tuning (#58163, #58023, #57996)
• Sample finalized partitions randomly to avoid lens effect (#58456)
• Allow file extensions starting with '.' (#58339)
• Set default file_extensions for read_parquet (#56481)
• URL decode values in parse_hive_path (#57625)
• Streaming partition enforces row_num per block (#57984)
• Streaming repartition combines small blocks (#58020)
• Lower DEFAULT_ACTOR_MAX_TASKS_IN_FLIGHT_TO_MAX_CONCURRENCY_FACTOR to 2 (#58262)
• Set udf-modifying-row-count default to false (#58264)
• Cache PyArrow schema operations (#58583)
• Explain optimized plans (#58074)
• Ranker interface (#58513)

🔨 Fixes:

• Fixed renamed columns to be appropriately dropped from output (#58040, #58071)
• Fixed handling of renames in projection pushdown (#58033, #58037)
• Fixed broken LogicalOperator abstraction barrier in predicate pushdown rule (#58683)
• Fixed file size ordering in download partitioning with multiple URI columns (#58517)
• Fixed HTTP streaming file download by using open_input_stream (#58542)
• Fixed expression mapping for Pandas (#57868)
• Fixed reading from zipped JSON (#58214)
• Fixed MCAP datasource import for better compatibility (#57964)
• Avoid slicing block when total_pending_rows < target (#58699)
• Clear queue for manually marked execution_finished operators (#58441)
• Add exception handling for invalid URIs in download operation (#58464)
• Fixed progress bar name display (#58451)

📖 Documentation:

• Documentation for Ray Data metrics (#58610)
• Simplify and add Ray Data LLM quickstart example (#58330)
• Convert rST-style to Google-style docstrings (#58523)

🏗 Architecture:

• Removed stats update thread (#57971)
• Refactor histogram metrics (#57851)
• Revisit OpResourceAllocator to make data flow explicit (#57788)
• Create unit test directory for fast, isolated tests (#58445)
• Dump verbose ResourceManager telemetry into ray-data.log (#58261)

Ray Train

🎉 New Features:

• Result::from_path implementation in v2 (#58216)

💫 Enhancements:

• Exit actor and log appropriately when poll_workers is in terminal state (#58287)
• Set JAX_PLATFORMS environment variable based on ScalingConfig (#57783)
• Default to disabling Ray Train collective util timeouts (#58229)
• Add SHUTTING_DOWN TrainControllerState and improve logging (#57882)
• Improved error message when calling training function utils outside Ray Train worker (#57863)
• FSDP2 template: Resume from previous epoch when checkpointing (#57938)
• Clean up checkpoint config and trainer param deprecations (#58022)
• Update failure policy log message (#58274)

📖 Documentation:

• Ray Train Metrics documentation page (#58235)
• Local mode user guide (#57751)
• Recommend tree_learner="data_parallel" in examples for distributed LightGBM training (#58709)

Ray Serve

🎉 New Features:

• Custom request routing with runtime environment support. Users can now define custom request router classes that are safely imported and serialized using the application's runtime environment, enabling advanced routing logic with custom dependencies. (#56855)
• Custom autoscaling policies with enhanced logging. Deployment-level and application-level autoscaling policies now display their custom policy names in logs, making it easier to debug and monitor autoscaling behavior. (#57878)
• Audio transcription support in vLLM backend. Ray Serve now supports transcription tasks through the vLLM engine, expanding multimodal capabilities. (#57194)
• Data parallel attention public API. Introduced a public API for data parallel attention, enabling efficient distributed attention mechanisms for large-scale inference workloads. (#58301)
• Route pattern tracking in proxy metrics. Proxy metrics now expose actual route patterns (e.g., /api/users/{user_id}) instead of just route prefixes, enabling granular endpoint monitoring without high cardinality issues. Performance impact is minimal (~1% RPS decrease). (#58180)
• Replica dependency graph construction. Added list_outbound_deployments() method to discover downstream deployment dependencies, enabling programmatic analysis of service topology for both stored and dynamically-obtained handles. (#58345, #58350)
• Multi-dimensional replica ranking. Introduced ReplicaRank schema with global, node-level, and local ranks to support advanced coordination scenarios like tensor parallelism and model sharding across nodes. (#58471, #58473)
• Proxy readiness verification. Added a check to ensure proxies are ready to serve traffic before serve.run() completes, improving deployment reliability. (#57723)
• IPv6 socket support. Ray Serve now supports IPv6 networking for socket communication. (#56147)

💫 Enhancements:

• Selective throughput optimization flag overrides. Users can now override individual flags set by RAY_SERVE_THROUGHPUT_OPTIMIZED without manually configuring all flags, improving flexibility for performance tuning. (#58057)
• OpenTelemetry metrics enabled by default. Ray now uses OpenTelemetry as the default metrics backend, with updated metric names (ray_serve_*) and improved observability infrastructure. (#56432)
• Cleaner long-poll communication. Removed actor handles from RunningReplicaInfo objects passed in long-poll updates, avoiding complex reference counting patterns. (#58174)
• Improved replica config handling. Excluded IMPLICIT_RESOURCE_PREFIX from ReplicaConfig.ray_actor_options to prevent internal resource annotations from leaking into user-visible configurations. (#58275)
• Custom autoscaling telemetry. Added telemetry tracking for custom autoscaling policy usage. (#58336)
• Proxy target group control. Added from_proxy_manager argument to get_target_groups() for finer control over returned routing targets. (#57620)

🔨 Fixes:

• Fixed default deployment name in async inference. Corrected the default deployment name which was changed to _TaskConsumerWrapper during async inference implementation. (#57664)
• Fixed proxy location handling in CLI and Python API. serve run now respects proxy_location from config files instead of hardcoding EveryNode, and serve.start() no longer defaults to HeadOnly when http_options are provided without an explicit location. (#57622)
• Fixed deprecated Stable Diffusion model in example. Updated documentation example to use a current model after stabilityai/stable-diffusion-2 was deprecated on Hugging Face. (#58609)

📖 Documentation:

• KV-cache offloading user guide. Added comprehensive documentation for KV-cache offloading in LLM deployments. (#58025)
• Model loading documentation. Documented best practices and options for loading models in Ray Serve. (#57922)
• Cross-node tensor/pipeline parallelism examples. Added examples and documentation for running TP/PP across multiple nodes. (#57715)
• Data parallel attention documentation. Created user guide for data parallel attention with architecture diagrams. (#58301, #58543)
• Custom autoscaling policy examples. Added missing imports and improved clarity in autoscaling policy examples. (#57896, #58170)
• Async inference documentation improvements. Added notes about task consumer replica configurations and fixed the end-to-end example. (#58493)
• Callback documentation. Added documentation for using callbacks in Ray Serve. (#58713)
• Monitoring and troubleshooting improvements. Enhanced monitoring section with links to Anyscale troubleshooting resources. (#58472)
• Minor documentation fixes. Fixed spelling errors and improved docstring alignment. (#58172, #58233)

🏗 Architecture refactoring:

• Replica rank management refactoring. Extracted generic RankManager class with type-safe ReplicaRank representation, creating a cleaner foundation for future multi-level rank support. (#58471, #58473)

Ray Tune

💫 Enhancements:

• Updated jobs test to use tune module (#57995)
• Add pydantic to Ray Tune requirements (#58354)

RLlib

🎉 New Features:

• Support for vectorize modes in SingleAgentEnvRunner.make_env (#58410)
• Support for composed spaces in Offline RL (#58594)
• Enhanced support for complex observations in SingleAgentEpisode (#57017)
• Prometheus metrics support for selected components (#57932)

💫 Enhancements:

• Improve test_single_agent_env_runner to prevent flaky tests (#58397)
• LINT improvements with enabled ruff imports across multiple modules (#56737, #56734, #56741, #56742, #56744, #56746)

🔨 Fixes:

• Resolve bug that fails to propagate model_config to MultiAgentRLModule instances (#58243)
• Fixed access to self._minibatch_size (#58595)
• Broken restore from remote - Add missing FileSystem argument (#58324)
• Fixed deterministic sampling and training documentation link (#58494)
• Corrected typo in pyspiel import error message (#54618)

📖 Documentation:

• Add reinforcement learning example illustrating GPU-to-GPU RDT and GRPO (#57961)

Ray Core

🎉 New Features:

• Token-based authentication across all Ray components (#58046, #58047, #58176, #58209, #58276, #58281, #58308, #58333, #58368, #58395, #58405, #58408, #58424, #58557, #57835, #58566, #58591)
• OpenTelemetry enabled by default for improved observability (#56432)
• Fallback strategy scheduling logic (#56369)
• TPU utility functions to support slice placement groups (#56723)
• Exponential backoff for retryable gRPCs (#56568)
• Option for in-flight RPC failure injection (#58512)
• Release test to simulate network transient errors via iptables (#58241)
• Nightly release test with cross-AZ fault injection (#57579)
• Owned object spill metrics (#57870)
• Monitoring in raylet for resource view (#58382)
• IPv6 support for sockets (#56147)

💫 Enhancements:

• Fault-tolerant RPCs: KillActor, CancelRemoteTask, NotifyGCSRestart, and ReleaseUnusedBundles (#57648, #57945, #57965)
• Use graceful actor shutdown when GCS polling detects actor ref deleted (#58605)
• Use graceful shutdown path when actor OUT_OF_SCOPE (del actor) (#57090)
• Improved actor kill logs (#58544)
• Scheduling detached actor with placement group not recommended (#57726)
• Better handling of detached actor restarts (#57931)
• Enhanced ray.get thread safety (#57911)
• Making concurrent ray.get requests for the same object thread-safe (#58606)
• Move request ID creation to worker to address plasma get perf regression (#58390)
• Make GlobalState lazy initialization thread-safe (#58182)
• Reporter agent can get PID via RPC to raylet (#57004)
• Add tee logging for subprocess exit codes in ray start --block (#57982)
• Add entrypoint log for jobs (#58300)
• Cleaner error message for exceeding list actors limit (#58255)
• Clean up NODE_DIED task error message (#58638)
• Improved histogram metrics midpoint calculation (#57948)
• Migrated from STATS to metric interface in RPC components (#57926)
• Kill STATS in core worker component (#58060)
• Kill STATS in object manager component (#57974)
• Improve scheduler_placement_time_s metric (#58217)
• Refactor OpenTelemetry environment variable handling (#57910)
• Add option to disable OpenTelemetry SDK error logs (#58257)
• Improved cgroups support (#57776, #57864, #57731, #58017, #58028, #58059, #58064, #58577)
• Use GetNodeAddressAndLiveness in raylet client pool (#58576)
• Ray Direct Transport improvements with NIXL integration (#57671, #58550, #58548, #56783, #58263)
• Fix symmetric-run (#58337)
• Make worker connection timeout parameters configurable (#58372)
• Define env for controlling UVloop (#58442)
• Allow 60 seconds for dashboard to start (#58341)
• Report driver stats (#58045)
• Fix idle node termination on object pulling (#57928)
• Check if temp_dir is subdir of virtualenv to prevent runtime virtualenv problems (#58084)

🔨 Fixes:

• Fixed use-after-free in RayletClient (#58747)
• Fixed deadlock when cancelling stale requests on in-order actors (#57746)
• Fixed "RayEventRecorder::StartExportingEvents() should be called only once" error (#57917)
• Fixed raylet shutdown races (#57198)
• Fixed incorrect usage of gRPC streaming API in ray syncer (#58307)
• Fixed log monitor seeking bug after log rotation (#56902)
• Fixed idempotency issues in RequestWorkerLease for scheduled leases (#58265)
• Fixed RAY_CHECK(inserted) inside reference counter (#58092)
• Fixed static type hints for ActorClass when setting options (#58439)
• Fixed exception type for accelerator ID visibility check (#58269)
• Fixed transport type handling in DAG node initialization (#57987)
• Fixed RAY_NODE_TYPE_NAME handling when autoscaler is in read-only mode (#58460)
• Ensure client_call_manager_ outlives metrics_agent_client_ in core worker (#58315)
• Fixed header validation in dashboard tests (#58648)
• Validation of Ray-on-Spark-on-YARN mode to enable it to run (#58335)

📖 Documentation:

• Fix pattern_async_actor demo typo (#58486)
• Add limitations of RDT documentation (#58063)
• Add actor+job+node event to ray event export documentation (#57930)
• Remove implementation details from get_runtime_context docstring (#58212)
• Improved monitoring section with links (#58472)

🏗 Architecture:

• Refactor ActorInfoAccessor in gcs_client to be mockable (#57241)
• Refactor reference_counter out of memory store and plasma store (#57590)
• Remove reference counter mock for real reference counter in testing (#57178)
• Split raylet cython file into multiple files (#56575)
• Move ray_syncer to top level directory (#58316)
• Move python_callbacks to common (#57909)
• Consolidate find_free_port to network_utils (#58304)
• Implement event merge logic at export time (#58070)
• Feature flag for enabling ray export event (#57999)
• Add comments explaining ray_syncer_ channels in Raylet (#58342)
• Integration tests for task event generation (#57636)

Dashboard

💫 Enhancements:

• Added percentage usage graphs for resources (#57549)
• Sub-tabs with full Grafana dashboard embeds on Metrics tab (#57561)
• Added queued blocks to operator panels (#57739)
• Improved operator metrics logging (#57702)
• Make do_reply accept status_code instead of success bool (#58384)
• Add denial of fetch headers (#58553)

🔨 Fixes:

• Fixed broken Ray Data per node metrics due to unsupported operator filter (#57970)
• Filtered out ANSI escape codes from logs (#53370)

📖 Documentation:

• Expose dashboard URL when deploying on Yarn using Skein (#57793)

Autoscaler + KubeRay

🎉 New Features:

• KubeRay autoscaling support with top-level Resources and Labels fields (#57260)
• Bundle label selector support in request_resources SDK (#54843)

💫 Enhancements:

• Azure VM launcher release test (#57921)
• Azure CLI added to base-extra image (#58012)

📖 Documentation:

• Label selector guide (#58157)
• Add minimum version requirement on kai-scheduler (#58161)
• Mention RayJob gang scheduling for Yunikorn (#58375)
• Add Volcano RayJob gang scheduling example (#58320)
• Add KAI scheduler integration documentation (#54857)
• Kuberay sidecar mode (#58273)
• Update RayJob documentation with new DeletionStrategy (#58306)
• Add guidance for RayService initialization timeout (#58238)
• Update version to 1.5.0 (#58452)
• Add output example of CLI commands (#58078)
• Fix invalid syntax in label_selector (#58352)

Thank You to all the Contributors! @marosset, @curiosity-hyf, @bveeramani, @Future-Outlier, @saihaj, @ZacAttack, @ArthurBook, @crypdick, @Aydin-ab, @elliot-barn, @Kunchd, @justinvyu, @jjyao, @gangsf, @sunsetxh, @Daraan, @justinyeh1995, @MatthewCWeston, @kyuds, @daiping8, @sauravvenkat, @omatthew98, @CowKeyMan, @morotti, @israbbani, @goutamvenkat-anyscale, @fscnick, @Zakelly, @xyuzh, @kouroshHakha, @owenowenisme, @Qiaolin-Yu, @czgdp1807, @shen-shanshan, @wph95, @iamjustinhsu, @MengjinYan, @jugalshah291, @Yicheng-Lu-llll, @ryanaoleary, @nadongjun, @xinyuangui2, @ideal, @my-vegetable-has-exploded, @lucaschadwicklam97, @tianyi-ge, @ahao-anyscale, @abrarsheikh, @Blaze-DSP, @rueian, @thomasdesr, @CaiZhanqi, @harshit-anyscale, @jeffreyjeffreywang, @TimothySeah, @codope, @sampan-s-nayak, @andrewsykim, @xingsuo-zbz, @aslonnie, @OneSizeFitsQuorum, @ryankert01, @Sparks0219, @soffer-anyscale, @akyang-anyscale, @alanwguo, @chrisfellowes-anyscale, @richo-anyscale, @alexeykudinkin, @JasonLi1909, @ruisearch42, @EkinKarabulut, @MarcoGorelli, @SolitaryThinker, @srinathk10, @dayshah, @richardliaw, @pseudo-rnd-thoughts, @win5923, @axreldable, @matthewdeng, @ArturNiederfahrenhorst, @can-anyscale, @khluu, @landscapepainter, @kevin85421, @seanlaii, @edoakes, @nrghosh, @eicherseiji, @Artimislyy, @cem-anyscale, @coqian, @chiayi, @liulehui

• Reuse previous metadata if transferring the same tensor list with nixl (https://github.com/ray-project/ray/pull/58309)

Release Highlights

Ray Train:

• Ray Train v2 is now enabled by default! Ray Train v2 provides usability and stability improvements, as well as new features. For more details, see the REP and Migration Guide. To disable Ray Train v2, set the environment variable RAY_TRAIN_V2_ENABLED=0.

Ray Serve:

• Application-level autoscaling: Introduces custom autoscaling policies that operate across all deployments in an application, enabling coordinated scaling decisions based on aggregate metrics. This is a significant advancement over per-deployment autoscaling, allowing for more intelligent resource management at the application level.
• Enhanced autoscaling capabilities with replica-level metrics: Wires up AutoscalingContext with total_running_requests, total_queued_requests, and total_num_requests, plus adds support for min, max, and time-weighted average aggregation functions. These improvements give users fine-grained control to implement sophisticated custom autoscaling policies based on real-time workload metrics.

Ray Libraries

Ray Data

🎉 New Features:

• Added enhanced support for Unity Catalog integration (#57954, #58049)
• New expression evaluator infrastructure for improved query optimization (#57778, #57855)
• Support for SaveMode in write operations (#57946)
• Added approximate quantile aggregator (#57598)
• MCAP datasource support for robotics data (#55716)
• Callback-based stat computation for preprocessors and ValueCounter (#56848)
• Support for multiple download URIs with improved error handling (#57775)

💫 Enhancements:

• Improved projection pushdown handling with renamed columns (#58033, #58037, #58040, #58071)
• Enhanced hash-shuffle performance with better retry policies (#57572)
• Streamlined concurrency parameter semantics (#57035)
• Improved execution progress rendering (#56992)
• Better handling of empty columns in pandas blocks (#57740)
• Enhanced support for complex data types and column operations (#57271)
• Reduced memory usage with improved streaming generator backpressure (#57688)
• Enhanced preemption testing and utilities (#57883)
• Improved Download operator display names (#57773)
• Better handling of variable-shaped tensors and tensor columns (#57240)
• Optimized aggregator execution with out-of-order processing by default (#57753)

🔨 Fixes:

• Fixed renamed columns to be appropriately dropped from output (#58040, #58071)
• Fixed handling of renames in projection pushdown (#58033, #58037)
• Fixed vLLMEngineStage field name inconsistency for images (#57980)
• Fixed driver hang during streaming generator block metadata retrieval (#56451)
• Fixed retry policy for hash-shuffle tasks (#57572)
• Fixed prefetch loop to avoid blocking on fetches (#57613)
• Fixed empty projection handling (#57740)
• Fixed errors with concatenation of mixed pyarrow native and extension types (#56811)

📖 Documentation:

• Updated document embedding benchmark to use canonical Ray Data API (#57977)
• Improved concurrency-related documentation (#57658)
• Updated preprocessing and data handling examples

Ray Train

🎉 New features

• Turn on Train v2 by default (#57857)
• Top-level ray.train aliases for public APIs (#57758)

💫 Enhancements

• Raise clear errors when mixing v1/v2 APIs (#57570)
• JAX backend: add jax.distributed.shutdown() for JaxBackend (#57802)
• Update TrainingFailedError module (#57865)
• Improve deprecation handling when ray.train methods are called from ray.tune (#57810)
• Enable deprecation warnings for legacy XGBoost/LightGBM trainers (#57280)

🔨 Fixes

• Fix ControllerError triggered by after_worker_group_poll_status errors (#57869)
• Fix iter_torch_batches use of ray.train.torch.get_device outside Train (#57816)
• Fix exception-queue race condition in ThreadRunner (#57249)

📖 Documentation

• Add validation and details to checkpoint docs (#57065)

🏗 Architecture / tests

• Enable Train v2 across test suites; migrate remaining tests and isolate/disable stragglers (#56868, #57256, #57534, #57722, #57764)
• Isolate circular-dependency tests and resolve circular imports (#57710, #56921)
• Replace Checkpoint Manager Pydantic v2 APIs with v1 (#57147)
• Bump test timeouts (test_util, torch_trainer) (#57939, #57873)

Ray Tune

💫 Enhancements:

• Updated release tests to import from tune (#57956)
• Better integration with Train V2 backend

Ray Serve

🎉 New Features:

• Application-level autoscaling. Introduces support for custom autoscaling policies that operate across all deployments in an application, enabling coordinated scaling decisions based on aggregate metrics. (#57535, #57548, #57637, #57756)
• Autoscaling metrics aggregation functions. Adds support for min, max, and time-weighted average aggregation over timeseries data, providing more flexible autoscaling control. (#56871)
• Enhanced autoscaling context with replica-level metrics. Wires up AutoscalingContext constructor arguments to expose total_running_requests, total_queued_requests, and total_num_requests for use in custom autoscaling policies. (#57202)
• Multiple task consumers in a single application. Ray Serve applications can now run multiple task consumer deployments concurrently. (#56618)

💫 Enhancements:

• Reconfigure invoked on replica rank changes. The reconfigure method now receives both user_config and rank parameters when ranks change, enabling replicas to adapt their configuration dynamically. (#57091)
• Celery adapter configuration improvements. Added default serializer and new configuration fields to enhance Celery integration flexibility. (#56707)
• AutoscalingContext promoted to public API. The autoscaling context is now officially part of the public API with comprehensive documentation. (#57600)
• Async inference telemetry. Added telemetry tracking to monitor the number of replicas using asynchronous inference. (#57665)
• Rank logging verbosity reduced. Changed seven rank-related INFO logs to DEBUG level, reducing log noise during normal operations. (#57831)
• Controller logging optimized. Removed expensive debug logs from the controller that were costly in large clusters. (#57813)

🔨 Fixes:

• Max constructor retry count test fixed for Windows. Adjusted test resource requirements to account for Windows process creation overhead compared to Linux forking. (#57541)
• Streaming test stability improvements. Added synchronization mechanisms to prevent chunk coalescing and rechunking, eliminating test flakiness. (#57592, #57728)
• Autoscaling test deflaking. Fixed race conditions in application-level autoscaling tests and removed flaky min aggregation test scenario. (#57784, #57967)
• State API usage test corrected. Fixed a unit test that was broken but not running in CI. (#56948)
• Controller recovery logging condition fixed. Updated test condition to properly verify debug and JSON logs after controller recovery. (#57568)

📖 Documentation:

• Custom autoscaling documentation. Added comprehensive guide for implementing custom autoscaling policies with examples and best practices. (#57600)
• Replica ranks documentation. Documented the replica rank feature, including how ranks are assigned and how to use them in reconfigure methods. (#57649)
• Application-level autoscaling guide. Added documentation explaining how to configure and use application-level autoscaling policies. (#57756)
• Autoscaling documentation improvements. Updated serve autoscaling docs with clearer explanations and examples. (#57652)
• Performance flags documentation. Documented performance-related configuration flags for Ray Serve. (#57845)
• Metrics documentation fix. Corrected ray_serve_deployment_queued_queries metric name discrepancy in documentation. (#57629)
• AutoscalingContext import added to examples. Fixed missing import statement in custom autoscaling policy example. (#57876)
• App builder guide typo corrected. Fixed command syntax error in typed application builder example. (#57634)
• Celery filesystem broker note. Added warning about using filesystem as a broker in Celery workers. (#57686)
• Async inference alpha stage warning. Added notice that async inference is in alpha stage. (#57268)

🏗 Architecture refactoring:

• Autoscaling control moved to application state. Migrated autoscaling control loop from deployment state to application state, preparing for application-level autoscaling. (#57548)
• Async capability enum removed. Cleaned up unused async capability enum from codebase. (#57666)

Ray Serve/Data LLM

🎉 New Features:

• Updated vLLM to 0.11.0 and Nixl to 0.6.0 (#57201)
• Video processor support for multimodal pipelines (#56785)
• Enhanced callback API for engine customization (#57257)
• Unified and extended builder configuration for LLM deployments (#57724)

💫 Enhancements:

• Protocol-based typing improvements and cleaner inheritance structure (#57743)
• Better engine metrics enabled by default (#57615)
• Simplified NIXL dependency management in ray-llm images (#57706)
• Per-stage map kwargs for LLM processor preprocessing/postprocessing (#57826)
• Improved architecture documentation (#57830)
• Better code structure alignment with architectural design (#57889)
• Enhanced multimodal support with Deepseek compatibility (#56906)

🔨 Fixes:

• Fixed NIXL limitations with proper exception handling (#58159)
• Improved runai_streamer for vLLM 0.10.2+ integration (#56906)

📖 Documentation:

• Added comprehensive architecture documentation for Ray Serve LLM (#57830)
• Reorganized LLM documentation with improved navigation (#57787)
• Added benchmark page for performance reference (#57960)
• Converted quick-start guide to MyST Markdown (#57782)
• Better organization of Ray Serve LLM documentation (#57181?)

RLlib

🎉 New Features:

• Prometheus metrics support for selected RLlib components (#57932)
• Enhanced support for complex observations in SingleAgentEpisode (#57017)

💫 Enhancements:

• LINT improvements with enabled ruff imports for rllib/utils (#56737)
• Better type hints for learner_connector (#57673)
• Improved throughput metrics to avoid biasing (#57215)

🔨 Fixes:

• Fixed segment_tree.py edge case (#57599)
• Fixed small bug in type hints (#57673)

Ray Core

🎉 New Features:

• Enhanced Ray Direct Transport (RDT) with improved NIXL integration and garbage collection (#57671, #57603, #58159)
• Cgroups support improvements with better system resource management (#57776, #57864, #57731, #58017, #58028, #58064)
• Fault-tolerant RPC improvements for better distributed reliability (#57786, #57861)
• Exponential backoff for retryable gRPCs (#56568)

💫 Enhancements:

• Migrated from STATS to metric interface in RPC components (#57926)
• Improved histogram metrics midpoint calculation (#57948)
• Made FreeObjects non-fatal for better error handling (#57550)
• Enhanced ReleaseUnusedBundles fault tolerance (#57786)
• Made DrainRaylet and ShutdownRaylet fault tolerant (#57861)
• Better error handling for metric and event exporter agent (#57925)
• Improved raylet shutdown process and file organization (#57817)
• Reporter agent can now get PID via RPC to raylet (#57004)
• Enhanced ray.get thread safety (#57911)
• Configurable proto naming during event JSON conversion (#57705)
• Better handling of detached actor restarts (#57931)
• Improved lease rescheduling in local lease manager during node draining (#57834)

🔨 Fixes:

• Fixed "RayEventRecorder::StartExportingEvents() should be called only once" error (#57917)
• Fixed deadlock when cancelling stale requests on in-order actors (#57746)
• Fixed raylet shutdown races (#57198)
• Fixed log monitor seeking bug after log rotation (#56902)
• Deflaked multiple test suites for better CI reliability
• Fixed various memory and resource management issues
• Better handling of actor and task failures

📖 Documentation:

• Added JaxTrainer API overview to Ray docs (#57182)
• Fixed various typos and documentation issues
• Updated autoscaling and system configuration guides
• Enhanced SLURM documentation with symmetric-run support (#56775)

🏗 Architecture refactoring:

• Dashboard API server subprocesses moved into system cgroup (#57864)
• Driver moved into workers cgroup for better isolation (#57776)
• Improved worker-raylet interface separation (#57804)
• Better plasma store provider architecture

Dashboard

💫 Enhancements:

• Added percentage usage graphs for resources (#57549)
• Introduced sub-tabs with full Grafana dashboard embeds on Metrics tab (#57561)
• Added queued blocks to operator panels (#57739)
• Improved operator metrics logging for better clarity (#57702)
• Better filtering and display in job lists

🔨 Fixes:

• Fixed filtering issue in job list (#56946)
• Fixed incomplete card content on overview page (#56947)
• Filtered out ANSI escape codes from logs (#53370)

Autoscaler

🎉 New Features:

• KubeRay autoscaling support with top-level Resources and Labels fields (#57260)
• Bundle label selector support in request_resources SDK (#54843)
• Application Gateway for Containers as ingress for Ray clusters on Azure

💫 Enhancements:

• Azure improvements: Cleaning up extra resources (MSI, VNET, NSG) during cluster teardown (#57610)
• Updated defaults for Azure cluster templates (#57716)
• Better availability zone support for Azure node pools (#55532)
• Hello world release tests for Azure and GCE (#57597, #57695)
• Improved cluster resource state handling to fix over-provisioning (#57130)

🔨 Fixes:

• Fixed autoscaler state synchronization issues (#57010)
• Better handling of node state information (#57130)
• Improved timeout handling for patch requests (#56605)

Thank you to everyone who contributed to this release! Special thanks to all the contributors who helped make Ray 2.51.0 possible through bug fixes, features, documentation improvements, and testing efforts.

Ray Core: Fix deadlock when cancelling stale requests on in-order actors (#57746)

Release Highlights

Ray Data: This release offers many updates to Ray Data, including:

• The default shuffle strategy is now changed from sort-based to hash-based. This will result in much lower peak memory usage and improved shuffle performance for aggregations.
• We’ve added a new expression API enables predicate-based filtering, UDF transformations with with_column, and column aliasing for more powerful data transformations
• Ray Data LLM has a number of new enhancements for multimodal data pipelines, including multi-node tensor and pipeline parallelism support per replica and ability to share vLLM engines across processors.

Ray Core:

Alpha release of Ray Direct Transport (formerly GPU objects) - simply enable it by adding the tensor_transport parameter to the existing native Ray Core API. This keeps GPU data in GPU memory until a transfer is needed, avoiding expensive serialization and copies to and from the Ray object store. It uses efficient data transports such as collective communication libraries (GLOO or NCCL) or point-to-point RDMA (via NVIDIA’s NIXL) to transfer data directly between devices, including both CPUs and GPUs.

Ray Train:

Local mode support for multi-process training with torchrun, enhanced checkpoint management with new upload modes and validation functions

Ray Serve:

• Async Inference alpha release - New Ray Serve APIs for supporting long-running asynchronous inference tasks, such as for video or large document processing. Includes capabilities for using different message brokers, adapters like celery and DLQ.
• Support for replica ranks - Replica level ranks are added for supporting large-model inference use-cases such as wide Data Parallel and Expert Parallel setups.
• FastAPI factory pattern support - Enables using FastAPI plugins that are not serializable via cloudpickle.
• Throughput optimizations - Enable these using the RAY_SERVE_THROUGHPUT_OPTIMIZED environment variable.

RLLib: Add StepFailedRecreateEnv exception for users with unsatisfiable environments

Ray Serve/Data LLM:

Improvements to multi node serving, loading models from remote storages, and sharing resources for efficiency (fractional gpus, sharing gpus on a data pipeline with shared stages)

Ray Libraries

Ray Data

🎉 New Features:

• Expression and Filtering API: New expression API enables predicate-based filtering, UDF transformations with with_column, and column aliasing for more powerful data transformations (#56716, #56313, #56550, #55915, #55788, #56193, #56596)
• Added support for projection pushdown into Parquet reads (#56500)
• New download expression enables efficient loading of data from columns containing URIs with improved performance and error handling (#55824, #56462, #56294, #56852, #57146)
• New explain() API provides insights into dataset execution plans (#55482)
• Added streaming_train_test_split to avoid materialization for train/test splits (#56803)
• Ray Data LLM:
  • Enabled multi-node tensor and pipeline parallelism for LLM processing (#56779)
  • Added chat_template_kwargs parameter for customizing chat templates (#56490)
  • Added support for OpenAI's nested image URL format in multimodal pipelines (#56584)
  • vLLM engines can now be shared across sequential processors for better resource utilization (#55179)
• Enhanced Dataset.stats() output with input/output row counts per operator (#56040)
• Added new metrics for task duration, inputs per task, and output blocks (#56958, #56379)
• Time to first batch metric for better iteration performance monitoring (#55758)
• Added type-specific aggregators for numerical, categorical, and vector columns (#56610)
• Added fine-grained concurrency controls with max_task_concurrency and resource allocation options (#56370, #56381)

💫 Enhancements:

• Join and shuffle improvements:
  • Default shuffle strategy changed from sort-based to hash-based for better performance (#55510)
  • Improved groupby performance with sort-shuffle pull-based approach (#57014)
  • Improved join operations with new abstractions (#57022, #56945, #55759)
• Tensor type handling improvements:
  • Improved compatibility between PyArrow native types, extension types, and pandas Arrow dtypes (#57566, #57176, #57057)
  • Joins now supported with list/tensor non-key columns (#5648)
  • Enhanced support for variable-shaped tensor arrays with different dimensions (#57240, #56918, #56457)
  • Added serialization/deserialization for PyArrow Extension Arrays (#51972)
• Removing Parquet metadata fetching in ParquetDatasource (#56105)
• Resource requirements (num_cpus/gpus, memory) are now top-level parameters in most APIs for easier configuration (#56419)
• zip() operator now supports combining multiple datasets, not just pairs (#56524)
• Concurrency parameter now accepts tuples for more flexible configuration (#55867)
• Write operations now use iterators instead of accumulating blocks in memory (#57108)
• Reduced memory usage for OneHotEncoder (#56565)
• Reduced memory usage for schema unification (#55880)
• Eliminated unnecessary block copying and double execution of arrow conversions (#56569, #56793)
• Improved Parquet encoding ratio estimation (#56268)
• Enabled per-block limiting for Limit operator (#55239)
• Optimized schema handling with deduplication and removed unnecessary unification (#55854, #55926)
• Improved issue detection with event emission instead of just logs (#55717)
• Better metric organization and external queue metric handling (#55495, #56604)
• New backpressure policy based on downstream processing capacity (#55463)

🔨 Fixes:

• Fixed streaming executor to properly drain output queues (#56941)
• Improved resource management and reservation for operators (#56319, #57123)
• Fixed retry logic for hash shuffle operations (#57575)
• Fix split_blocks produce empty blocks (#57085)
• Initialize datacontext after setting src_fn_name in actor worker (#57117)
• Fix mongo datasource collStats invocation (#57027)
• Fixing empty projection handling in ParquetDataSource (#56299)
• Fix UnboundLocalError when read_parquet with columns and no partitioning (#55820)
• Fix high memory usage with FileBasedDatasource & ParquetDatasource when using a large number of files (#55978)
• [llm] Fixed LLM processor deployment with Ray Serve (#57061)
• [llm] Fixed multimodal image extraction when system prompts are absent (#56435)
• Ignore metadata for pandas block (#56402)
• Remove metadata for hashing + truncate warning logs (#56093)

📖 Documentation:

• Error in ray.data.groupby example in docs. (#57036)
• Update on ray.data.Dataset.map() type hints. (#52455)
• Small typo fix. (#56560)
• Fix a typo. (#56587)
• Fix documentation for new execution options resource limits assignment. (#56051)
• Fix broken code snippets in user guides. (#55519)
• Add Autoscaling Config for Context docs. (#55712)
• Make object store tuning tips consistent with other pages. (#56705)
• New example of how to perform batch inference with embedding models (#56027)

Ray Train

🎉 New Features:

• Local mode support for Ray Train V2
  • Add local mode support to Ray Train v2 (num_workers=0). (#55487)
  • Add PyTorch local mode support for multi-process training with torchrun. (#56218)
• Async checkpoint and validation for Ray Train
  • Add checkpoint_upload_mode to ray.train.report. (#55637)
  • Add checkpoint_upload_function to ray.train.report. (#56208)
  • Add validate_function and validate_config to ray.train.report. (#56360)
  • Add ray.train.get_all_reported_checkpoints method. (#54555)

💫 Enhancements:

• Ray Train V2 Migration
  • Implement BaseWorkerGroup for V1/V2 compatibility. (#57151)
• Train Controller is always actor + fix tune integration to enable this. (#55556)
• Refactor AcceleratorSetupCallback to use before_init_train_context. (#56509)
• Move collective implementations to train_fn_utils. (#55689)
• Ray Train Framework support enhancements
  • Add hf trainer support for dictionary of datasets. (#56484)
  • Add usage tag key for JaxTrainer. (#55887)
• Add Torch process group shutdown timeout. (#56182)
• Ray Train disables blocking get inside async warning. (#56757)
• ThreadRunner captures exceptions from nested threads. (#55756)
• Abort reconciliation thread catches ray.util.state.get_actor exception. (#56600)
• Ray Data Integration
  • Minor rework of get_dataset_shard. (#55825)
  • Create a deepcopy of the data context on the split coordinator process. (#56211)
  • Enable debug logging; fix default actor_locality_enabled. (#56632)
• Refactor call_with_retry into shared library and use it to retry checkpoint upload. (#56608)
• Remove Placement Group on Train Run Abort. (#56011)

🔨 Fixes:

• Fix LightGBM v2 callbacks for Tune only usage. (#57042)
• Ignore tensorflow test for py312. (#56244)
• Revising test_jax_trainer flaky test. (#56854)
• Fix test_jax_trainer imports. (#55799)
• Fix test_jax_trainer::test_minimal_multihost Flaky Test. (#56548)
• Disable drop_last flag to fix division by zero in torch dataloader baselines. (#56395)
• Preload a subset of modules for torch dataloader forkserver multiprocessing. (#56343)

📖 Documentation:

• Add checkpoint_upload_mode to checkpoint docs. (#56860)
• Add get_all_reported_checkpoints and ReportedCheckpoint to API docs. (#56174)
• Fix typo for Instantiating in ray train doc. (#55826)

🏗 Architecture refactoring:

• Release tests for ray train local mode. (#56862)
• Migrate tune_rllib_connect_test & tune_cloud_long_running_cloud_storage to ray train v2. (#56844)
• Add v2 multinode persistence release test. (#56856)
• Attach a quick checkpoint when reporting metrics. (#56718)
• Upgrade tune_torch_benchmark to v2. (#56804)
• Move tune_with_frequent_pausing to Ray Train v2 and tune_tests folder. (#56799)
• Migrate xgboost/lgbm benchmarks to train V2. (#56792)

Ray Tune

🎉 New Features:

• Trigger Checkpointing via Trial / Tuner Callback. (#55527)

💫 Enhancements:

• Improve _PBTTrialState for dev/debugging usage. (#56890)
• Enable Train V2 in Tune unit tests and examples. (#56816)
• Enable Train v2 in doc examples. (#56820)
• Reintroduce keras tune callback. (#57121)

🔨 Fixes:

• Increase tune checkpoint test latency threshold. (#56251)
• Remove a bunch of low-signal/redundant train/air/tune tests. (#56477)
• Remove tune_air_oom test. (#57089)

Ray Serve

🎉 New Features:

• Add tests and DLQ business logic for async inference. (#55608)
• Foundation work for aggregating metrics on controller. (#55568)
• Include custom metrics method and report to controller. (#56005)
• Add post scaling api. (#56135)
• Introduce deployment rank manager. (#55729)
• Integrated deployment ranks with deployment state. (#55829)
• Add rank and world size in replica context. (#55827)
• Added ssl to ray serve. (#55228)
• Custom parameter for downscaling to zero. (#56573)
• Add optional APIType filter to /api/serve/applications/ endpoint. (#56458)
• Make deployment retry configurable. (#56530)

💫 Enhancements:

• Aggregate autoscaling metrics on controller. (#56306)
• Update metrics_utils for future global metrics aggregation in controller. (#55568)
• Use deployment method in access logs for replicas. (#56829)
• Cache router metrics. (#55897)
• Allow same event loop handle shutdown from sync context. (#55551)
• Additional deps to start with prometheus. (#57155)
• Require prefix RAY_SERVE_ for env vars + value verification. (#55864)
• Record queued metrics on timeseries. (#57024)
• Add throughput opt env var for serve. (#55804)
• Fix None pending Request. (#54775)
• Omit unnecessary newlines in the config generated by serve build app:app. (#56609)
• Expose actor name for target group api. (#56738)

🔨 Fixes:

• Fix proxy lua dependency in dockerfile. (#57221)
• Fix non thread safe asyncio task creation in router. (#56124)
• Fix throughput optimized benchmarks. (#56173)
• Move ingress validation for multiple fastapi deployment into client. (#56706)
• Explicitly close choose_replicas_with_backoff async generator. (#56357)
• Fix buffered logging reusing request context. (#56094)
• Use default gc frequency for proxy. (#56511)
• Fixing deployment scoped custom autoscaling. (#56192)

📖 Documentation:

• Stable links for Ray serve. (#56241)
• Add document for using fastapi factory pattern in serve. (#56607)
• Add documentation for async inference (#56453)

🏗 Architecture refactoring:

• Add microbenchmark for throughput optimized configuration. (#55900)
• Only checkpoint controller state when it is confirmed that target state has changed (#55848)
• Proxy Actor Interface. (#56288)
• Allow ProxyActor to return true/false for health check. (#56660)

Ray Serve/Data LLM

🎉 New Features:

• Score API Integration for Serve LLM. (#55914)
• Add start/stop_profile method to LLMServer. (#55920)
• Add prefix cache hit rate to Serve LLM dashboard. (#55675)
• Configure aggregation interval for dashboard. (#56591)

💫 Enhancements:

• Bump vLLM to 0.10.2. (#56535)
• Vllm bump -> 0.10.1.1. (#56099)
• Refactor: Improve Deployment Builder Ergonomics and Code Organization. (#57181)
• Fix build_llm_processor for ServeDeploymentProcessor. (#57061)
• Allow setting data_parallel_size=1 in engine_kwargs. (#55750)
• Allow tuple for concurrency arg. (#55867)
• Fix multimodal image extraction when no system prompt is present. (#56435)
• Support azure and abfss in LLM config. (#56441)
• Support custom s3 endpoint when downloading models from remote. (#55458)
• Skip safetensor file downloads for runai streamer mode. (#55662)
• Support colocating local DP ranks in DPRankAssigner. (#55720)
• Adjust LLM engine timing logic. (#55595)
• Fixed DP DSV3 issues. (#55802)
• Gracefully return timeouts as HTTPException. (#56264)
• Remove upstreamed workarounds 1/3. (#54512)

🔨 Fixes:

• Changed LMCache dependency to use 0.3.3 to avoid regressions in the release test. (#56104)
• Fix doc test for Working with LLMs guide. (#55917)
• Fix sglang byod on release. (#55885)

📖 Documentation:

• Add gpt oss deployment example. (#56400)
• Add serve llm example to index page + other minor fix. (#56788)
• Example serve llm deployment. (#55819)
• Fix serve llm examples. (#56382)
• Docs: serve llm deployment examples refinement. (#56287)
• Add example of serving a VLLM model on fractional gpu. (#57197)
• Add main pytest code snippet to those tests that were missing it. (#57167)

RLlib

🎉 New Features:

• Add StepFailedRecreateEnv exception. (#55146)

💫 Enhancements:

• Add tags to envrunner calls, count in flight requests in ActorManager. (#56930, #56953)
• Add spaces in case only offline data is used. (#56141)
• Add Footsies environment and tests. (#55041)

🔨 Fixes:

• Fix failing env step in MultiAgentEnvRunner. (#55567)
• Fix Metrics/Stats lifetime count and throughput measurement for async remote actors. (#56047)
• Fixes Implementation of Shared Encoder. (#54571)
• Fix MetricsLogger/Stats throughput bugs. (#55696)

📖 Documentation:

• [RLlib] [DOC] Fix documentation typos and grammatical issues in RLlib docs (#56130)
• Update rllib-env.rst - typo. (#56140)
• Fixing typo in the RLlib documentation. (#55752)
• Fix formatting of class references. (#55764)

🏗 Architecture refactoring:

• Remove checkpoint release tests. (#57105)
• Remove long_running_apex test. (#57097)
• LINT: Enable ruff imports for multiple directories in rllib. (#56736)
• Upgrade g3 to g4 machine for aws release test. (#56248)

Ray Core

🎉 New Features:

• Alpha release of Ray Direct Transport
  • Support ray.put() and ray.get() with nixl in gpu objects. (#56146)
  • Support using ray.get with nixl to retrieve data from GPU object refs created by remote tasks. (#56559)
  • Support tensor transfer from outside owners of actors. (#56485)
  • Automatically enable tensor transport for the actor if any method specifies one. (#55324)
  • Support cpu tensor transfer with NIXL in GPU Objects. (#55793)
  • Handle multiple transfers of the same object to an actor. (#55628)
  • Support NIXL as tensor transport backend. (#54459)
  • Add a user-facing call to wait for tensor to be freed. (#55076)
  • Always write to GPUObjectStore to avoid _get_tensor_meta() from hanging indefinitely. (#55433)
  • Add warning when GPU object refs passed back to the same actor. (#55639)
  • Avoid triggering a KeyError by the GPU object GC callback for intra-actor communication. (#54556)
• Enable autoscaler v2 on clusters launched by the cluster launcher. (#55865)
• Ray Symmetric Run Script and ray symmetric-run command. (#55111, #56497)

💫 Enhancements:

• Ray Event Export
  • GCS AddEvent support. (#55528)
  • Actor event: add proto schema. (#56221)
  • Node event: add proto schema and send node events to the aggregator. (#56031, #56426)
  • Job event: add schema for driver job event and send job events to the aggregator. (#55032, #55213)
  • Emit actor events to Event aggregator. (#56617)
  • Export node event by default. (#56810)
• Security
  • Bind ray internal servers to the specified node ip instead of 0.0.0.0. (#55178, #55210, #55298, #55484)
  • Bind dashboard agent http server to localhost in addition to the node ip. (#55910)
  • Bind dashboard agent grpc to specified ip instead of 0.0.0.0. (#55732)
  • Bind runtime env agent and dashboard agent http server to specified ip instead of 0.0.0.0. (#55431)
• RPC network fault tolerance
  • Making core worker pub sub RPCs fault tolerant. (#56436)
  • Make RequestWorkerLease RPC Fault Tolerant. (#56191)
  • Making ReturnWorkerLease Idempotent. (#56073)
  • Making CancelWorkerLease RPC Fault Tolerant. (#56195)
  • Make Free Objects RPC Fault Tolerant. (#56293)
  • Make PinObjectIDs RPC Fault Tolerant. (#56443)
  • Make Unsubscribe Idempotent. (#57546)
  • Core Worker GetObjStatus GRPC Fault Tolerance. (#54567)
• Not overriding accelerator id env vars when num_accelerators is 0 or not set. (#54928)
• Migrate metric collection from opencensus to opentelemetry. (#53098, #53740)
• Add per worker process group and deprecate process subreaper in favor of cleanup using process group. (#56476)
• Add node_id validation in NodeAffinitySchedulingStrategy. (#56708)
• Add io_context metrics to gcs and raylet. (#55762)
• Modify RedisDelKeyPrefixSync to use the Redis SCAN command instead of KEYS. (#56907)
• Add error_type to job failures. (#55578)
• Add PID to structured logs for tasks and actors. (#55176)
• Log actor name when warning about excess queueing. (#57124)
• Output the error log on the driver side if the failed task will still retry. (#56472)
• Prometheus http service discovery API. (#55656)
• Add node ip in runtime env error message to improve debug observability. (#56837)
• Fallback unserializable exceptions to their string representation. (#55476)
• Introduce new exception type for un-pickleable exceptions. (#55878)
• Improve docs for custom serialization for exceptions + add test. (#56156)
• Add a warning when returning an object w/ num_returns=0. (#56213)
• Adding ability to specify availability zones for ray cluster node pools on Azure. (#55532)
• Query for supported Microsoft.Network/virtualNetworks API versions instead of relying on resource_client.DEFAULT_API_VERSION. (#54874)
• Loosen Ray self-dependency check to allow matching versions. (#57019)
• Add support for pip_install_options for pip. (#53551)
• Proper typing for ObjectRef. (#55566)

🔨 Fixes:

• Use subscription id from azure profile if not provided in config during AzureNodeProvider init. (#56640)
• Always create standard public IP addresses (basic sku is deprecated). (#57131)
• Fix: bug with config key pairs when launching worker nodes. (#57107)
• If azure cluster launcher keypair doesnt exist create one automatically + doc typo fix. (#54596)
• Fix "objects_valid" for the case that multiple instances of the same task are storing returns. (#54904)
• Fix objects_valid check failure with except from BaseException. (#55602)
• Preserve err type in case of task cancellation due to actor death. (#57538)
• Fix checking for uv existence during ray_runtime setup. (#54141)
• Prevent sending SIGTERM after calling Worker::MarkDead. (#54377)
• Fixed the bug where the head was unable to submit tasks after redis is turned on. (#54267)
• Fix possible race by checking node cache status instead of just subscription. (#54745)
• Fix get actor timeout multiplier. (#54525)
• Use a temporary file to share default worker path in runtime env. (#53653)
• Fix check fail when task buffer periodical runner runs before RayEvent is initialized. (#55249)
• Patch grpc with RAY_num_grpc_threads to control grpc thread count. (#54988)
• Fix HandleRefRemoved thread safety. (#56445)
• Fix error handling for plasma put errors. (#56070)
• Fix batching logic in CoreWorkerPlasmaStoreProvider::Get. (#56041)
• Fix RAY_CHECK failure during shutdown due to plasma store race condition. (#55367)
• Fix autoscaler RAY_CHECK when GcsAutoscalerStateManager is out of sync with NodeManager. (#57010)
• Fix bug where inflight requests are not taken into account by retryable. (#57142)
• A timeout should be set when submitting patch requests for autoscaler. (#56605)
• Fix the bug in memray regarding the default configuration of -o {output_file_path}. (#56732)
• Fixed the issue of RemoveActorNameFromRegistry being called repeatedly. (#54955)
• Fixed an issue where the command executed when use_podman=true and run_env=None was not prefixed with podman exec. (#56619)
• Fix data race when using async gpu to gpu transfer. (#57112)
• Retry + Make FreeActorObject idempotent. (#56447)
• Fix check crash on gpu obj free if driver knows actor is dead. (#56404)
• Handle system errors with a background monitor thread. (#56513)
• Fix GPU metrics. (#56009)
• Don't disconnect worker client on OBOD unless the worker is dead. (#57185)
• Prevent stale GET request being registered if its lease was cleared. (#56766)
• Drop messages received after ClientConnection::Close. (#56240)
• Fix cancel race that leads to RAY_CHECK it->second.submitted_task_ref_count > 0. (#56123)
• Reorder asyncio actor shutdown to terminate asyncio thread first. (#56827)
• Fix actor import error message for async actors. (#55722)
• Allow task manager access with submitter mutex + unify retry. (#56216)
• Fix bug in restore_from_path such that connector states are also restored on remote EnvRunners. (#54672)
• Fix S3 access issue in AKS. (#56358)
• Add S3 public bucket fallback to handle NoCredentialsError. (#56334)
• Fix ABFSS (Azure Blob File System Secure) protocol support problems during E2E test. (#56188)
• Ray cluster commands (up, attach, status, etc) updates to work on Windows. (#54982)
• Update cluster scheduler to handle label selector hard node id constraint. (#56235)

📖 Documentation:

• Added guide on using type hints with Ray Core. (#55013)
• Lifecycle of a task. (#55496)
• Add OSS Document for Task Events. (#56203)
• Fix Missing Events Issue in Task Events. (#55916)
• Add docs for asyncio and object mutability. (#56790)
• Update getting started and set up document for ray on vsphere. (#56954)
• Docfix - rst annotation showing up in render. (#57104)
• Add threading requirement to NodeProvider interface. (#56349)
• Add guidance for matching Ray and Python versions with uv envs. (#56597)
• Fix documentation typos, grammar, and terminology inconsistencies. (#56066, #56067, #56068, #56069, #56128, #56129, #56130, #56131, #56132, #56272, #56273, #56274, #56275, #56277, #56278, #56279)
• Update SLURM docs with symmetric-run. (#56775)
• Update Kueue integration documentation to include RayService & RayCluster support. (#56781)
• Application Gateway for Containers as ingress to access Ray Cluster. (#56574)
• Update DLAMI Information in aws.md. (#55702)

Dashboard

💫 Enhancements:

• Use pynvml for GPU metrics. (#56000)
• Default dashboard usability improvements. (#55620)
• Make Ray Train Dashboard Panel Ids Static. (#55559)
• Small fixes to Metrics Tab for kube-ray clusters. (#57149)
• Add metadata to indicate full dashboard embedding is supported. (#56077)
• Use ray node id instead of ip for profilinglink. (#55439)
• Fix grafana dashboard generation bug. (#56346)
• Catch OSError when detecting the GPU. (#56158)

🔨 Fixes:

• Removed references to a deleted Data metrics panel. (#55478)
• Fix typo in memory_utils and adjust display formatting for clarity. (#56217)

Ray Images

🎉 New Features:

• Add support for building and publishing ray-extra images. (#56543)
• Add ray-llm and ray-ml extra images. (#56800)
• Build ray-extra images for aarch64. (#56818)
• Add slim image to the image build matrix. (#55723)

💫 Enhancements:

• Add haproxy binary, for ray serve use. (#56845)
• Add ~/.local/bin to PATH in slim image. (#56920)
• Remove slim's dependency on normal bases. (#56544)
• Add label for ray version and commit. (#56493)
• Refactor apt package installation. (#55701)
• Allow using explicit base type. (#56545)
• Add extra-test stage in image building. (#55725)
• Add test rules for image building files. (#56554)
• Add ray-llm image type check. (#56542)
• Unify label and tag conventions. (#56189)
• GKE GPU compat paths: PATH, LD_LIBRARY_PATH (temporarily). (#55569)
• Stop publishing ray-ml images. (#57070)
• Stop building and releasing x86 osx wheels. (#57077)

📖 Documentation:

• Update latest Docker dependencies for 2.49.0 release. (#55966)
• Update latest Docker dependencies for 2.49.2 release. (#56760)

Wheels and images

💫 Enhancements:

• Use bazel run to generate files required for the wheel and testing (#55957, #56527, #56969, #56004, #55928)
• Ban click 8.3.0. (#56789)
• Upgrade protobuf to v4. (#54496)
• Add adlfs[abfs] into image (#56084)
• Upgrade boto3 to 1.29.x. (#56363)
• Upgrading orjson to 3.9.15. (#55972)
• Update spdlog to 15.3. (#56711)

Thanks!

Thank you to everyone who contributed to this release!

@alexeykudinkin, @richardliaw, @nrghosh, @ljstrnadiii, @Daraan, @kouroshHakha, @Bye-legumes, @kamil-kaczmarek, @jugalshah291, @sampan-s-nayak, @jjyao, @Evelynn-V, @gangsf, @omatthew98, @TimothySeah, @kshanmol, @goutamvenkat-anyscale, @axreldable, @jiangwu300, @simonsays1980, @400Ping, @JasonLi1909, @chuang0221, @weiliango, @Myasuka, @win5923, @liulehui, @khluu, @ok-scale, @eicherseiji, @tianyi-ge, @MengjinYan, @kevin85421, @Yevet, @orangeQWJ, @vie-serendipity, @edoakes, @wyhong3103, @israbbani, @vickytsang, @HassamSheikh, @acrewdson, @czgdp1807, @daiping8, @carolynwang, @thc1006, @jeffreyjeffreywang, @Stack-Attack, @Catch-Bull, @elliot-barn, @Levi080513, @BestVIncent, @dragongu, @jmajety-dev, @jcarlson212, @tohtana, @abrarsheikh, @crypdick, @Yicheng-Lu-llll, @ZacAttack, @justinvyu, @lk-chen, @alanwguo, @mcoder6425, @my-vegetable-has-exploded, @yancanmao, @arcyleung, @rjpower, @codope, @harshit-anyscale, @dayshah, @stephanie-wang, @KaisennHu, @ryanaoleary, @saihaj, @mattip, @rueian, @Kunchd, @pavitrabhalla, @owenowenisme, @Aydin-ab, @gvspraveen, @minerharry, @JackGammack, @jpatra72, @coqian, @zcin, @dstrodtman, @aslonnie, @ahao-anyscale, @GuyStone, @iamjustinhsu, @seanlaii, @ruisearch42, @akyang-anyscale, @ArturNiederfahrenhorst, @bveeramani, @OneSizeFitsQuorum, @xinyuangui2, @sb-hakunamatata, @22quinn, @Sparks0219, @sven1977, @snehachhabria, @dioptre, @nadongjun, @eric-higgins-ai, @marosset, @MatthewCWeston, @pcmoritz, @can-anyscale, @pimdh, @roshankathawate, @matthewdeng, @martinbomio, @GokuMohandas, @alimaazamat, @ali-corpo, @landscapepainter, @Qiaolin-Yu, @vaishdho1, @avigyabb, @srinathk10, @tannerdwood

There is no difference between 2.49.2 and 2.49.1, though we needed a patch version for other out of band reasons. To fill the awkward blankness, here is a haiku about Ray:

Summit drawing near Ray advances, step by step Scaling without end

• Ray Dashboard: Fix issue where GPU metrics are missing (#56006)
• Ray Data: Fixed regression in handling very large schemas (#56058)

Release Highlights

Ray Data:

• We’ve implemented a variety of performance enhancements, including improved actor/node autoscaling with budget-aware decisions; faster/more accurate shuffle accounting; reduced Parquet metadata footprint; and out-of-order execution for higher throughput.
• We’ve also implemented anti/semi joins, stratified train_test_split, and added Snowflake connectors.

Ray Core:

• Performance/robustness cleanups around GCS publish path and raylet internals; simpler OpenTelemetry flagging; new user-facing API to wait for GPU tensor free; plus assorted test/infra tidy-ups

Ray Train:

• We’ve introduced a new JaxTrainer with SPMD support for TPUs.

Ray Serve:

• Custom Autoscaling per Deployment Serve now supports user-defined autoscaling policies via AutoscalingContext and AutoscalingPolicy, enabling fine-grained scaling logic at the deployment level. This is part of a large effort where we are adding support for autoscaling based on custom metrics in Serve, see this RFC for more details.
• Async Inference (Initial Support): Ray Serve introduces asynchronous inference execution, laying the foundation for better throughput and latency in async workloads. Please see this RFC for more details.
• Major Performance Gains: This version of ray serve brings double digit % performance improvements both in throughput and latency. See release notes for more details.

Ray Serve/Data LLM:

• We’ve refactored Ray Serve LLM to be fully compatible with the default vllm serve and also now supports vLLM=0.10.
• We’ve added a prefix cache-aware router with PrefixCacheAffinityRouter for optimized cache utilization; dynamic cache management via reset prefix cache remote methods; enhanced LMCacheConnectorV1 with kv_transfer_config support.

Ray Libraries

Ray Data

🎉 New Features:

• Wrapped batch indices in a BatchMetadata object to make per-batch metadata explicit. (#55643)
• Added support for Anti/Semi Join types. (#55272)
• Introduced an Issue Detection Framework. (#55155)
• Added an option to enable out-of-order execution for better performance. (#54504)
• Introduced a StreamingSplit logical operator for DAG rewrite. (#54994)
• Added a stratify parameter to train_test_split. (#54624)
• Added Snowflake connectors. (#51429)
• Updated Hudi integration to support incremental query. (#54301)
• Added an Actor location tracker. (#54590)
• Added BundleQueue.has_next. (#54710)
• Made DEFAULT_OBJECT_STORE_MEMORY_LIMIT_FRACTION configurable. (#54873)
• Added Expression support & a with_columns API. (#54322)
• Allocate GPU resources in ResourceManager. (#54445)

💫 Enhancements:

• Decoupled actor and node autoscaling; autoscaling now also considers budget. (#55673, #54902)
• Faster hash-shuffle resource usage calculation; more accurate shuffle progress totals. (#55503, #55543)
• Reduced Parquet metadata storage usage. (#54821)
• Export API improvements: refresh dataset/operator state, sanitize metadata, and truncate exported metadata. (#55355, #55379, #55216, #54623)
• Metrics & observability: task metric improvements, external-buffer block-count metric, row-based metrics, clearer operator names in logs, single debug log when aggregators are ready. (#55429, #55022, #54693, #52949, #54483)
• Dashboard: added “Max Bytes to Read” panel/budget, panels for blocks-per-task and bytes-per-block, and streaming executor duration. (#55024, #55020, #54614)
• Planner/execution & infra cleanups: ExecutionResources and StatsManager cleanup, planner interface refactor, node trackers init, removed ray.get in _MapWorker ctor, removed target_shuffle_max_block_size. (#54694, #55400, #55018, #54665, #54734, #55158)
• Behavior/interop tweaks: map_batches defaults to row_modification=False and avoids pushing past limit; limited operator pushdown; prefetch for PandasJSONDatasource; use cloudpickle for Arrow tensor extension ser/des; bumped Arrow to 21.0; schema warning tone change. (#54992, #54457, #54667, #54831, #55426, #54630)
• Removed randomize-blocks reorder rule for more stable behavior. (#55278)

🔨 Fixes:

• AutoscalingActorPool now properly downscales after execution. (#55565)
• StatsManager handles StatsActor loss on disconnect. (#55163)
• Handle missing chunks key when Databricks UC query returns zero rows. (#54526)
• Handle empty fragments in sampling when num_row_groups=0. (#54822)
• Restored handling of PyExtensionType to keep compatibility with previously written datasets. (#55498)
• Prevent negative resource budget when concurrency exceeds the global limit; fixed resource-manager log calculation. (#54986, #54878)
• Default write_parquet warning removed; handled unhashable types in OneHotEncoding. (#54864, #54863)
• Overwrite mode now maps to the correct Arrow behavior for parallel writes. (#55118)
• Added back from_daft Arrow-version checks. (#54907)
• Pandas chained in-place assignment warning resolved. (#54486)
• Test stability/infra: fixed flaky tests, adjusted bounds and sizes, added additional release tests/chaos variants for image workloads, increased join test size, adjusted sorting release test to produce 1 GB blocks. (#55485, #55489, #54806, #55120, #54716, #55402, #54971)

📖 Documentation:

• Added a user guide for aggregations. (#53568)
• Added a code snippet in docs for partitioned writes. (#55002)
• Updated links to Lance documentation. (#54836)

Ray Train

🎉 New Features:

• Introduced JaxTrainer with SPMD support on TPUs (#55207)

💫 Enhancements:

• ray.train.get_dataset_shard now lazily configures dataset sharding for better startup behavior (#55230)
• Clearer worker error logging (#55222)
• Fail fast when placement group requirements can never be satisfied (#54402)
• New ControllerError surfaced and handled via failure policy for improved resiliency (#54801, #54833)
• TrainStateActor periodically checks controller health and aborts when necessary (#53818)

🔨 Fixes:

• Resolve circular import in ray.train.v2.lightning.lightning_utils (#55668)
• Fix XGBoost v2 callback behavior (#54787)
• Suppress a spurious type error (#50994)
• Reduce test flakiness: remove randomness and bump a data-integration test size (#55315, #55633)

📖 Documentation:

• New LightGBMTrainer user guide (#54492)
• Fix code-snippet syntax highlighting (#54909)
• Minor correction in experiment-tracking guide comment (#54605)

🏗 Architecture refactoring:

• Public Train APIs routed through TrainFnUtils for consistency (#55226)
• LoggingManager utility for Train logging (#55121)
• Convert DEFAULT variables from strings to bools (#55581)

Ray Tune

🎉 New Features:

• Add video FPS support to WandbLoggerCallback (#53638)

💫 Enhancements:

• Typing: reset_config now explicitly returns bool (#54581)
• CheckpointManager supports recording scoring metric only (#54642)

🔨 Fixes:

• Fix XGBoost v2 callback integration (#54787)
• Correct type for RunConfig.progress_reporter (#48439)

📖 Documentation:

• Minor fixes (#55125, #54942)

Ray Serve

🎉 New Features:

• Async inference support in Ray Serve (initial phase). Provides basic asynchronous inference execution, with follow-up work planned for failed/unprocessed queues and additional tests. #54824
• Per-deployment custom autoscaling controls. Introduces AutoscalingContext and AutoscalingPolicy classes, enabling user-defined autoscaling strategies at the deployment level. #55253
• Same event loop router. Adds option to run the Serve router in the same event loop as the proxy, yielding ~17% throughput improvement. #55030

💫 Enhancements:

• Async get_current_servable_instance(). Converts the FastAPI dependency to async def, removing threadpool overhead and boosting performance: 35% higher RPS and reduced latency. #55457
• Access log optimization. Cached contexts in request path logging improved request throughput by ~16% with lower average latency. #55166
• Batching improvements. Default batch wait timeout increased from 0.0s to 0.01s (10ms) to enable meaningful batching. #55126
• HTTP receive refactor. Cleaned up handling of replica-side HTTP receive tasks. #54543 / #54565
• Configurable replica router backoff. Added knobs for retry/backoff control when routing to replicas. #54723
• Autoscaling ergonomics. Marked per-deployment autoscaling metrics push interval config as deprecated for consistency. #55102
• Health check & env var safety. Introduced warnings for invalid/zero/negative environment variable values, with migration path planned for Ray 2.50.0. #55464, #54944
• Improved CLI UX. serve config now prints No configuration was found. instead of an empty string. #54767

🔨 Fixes:

• Removed brittle ray._private dependency usage. #55659
• HTTP route test fixes. Migrated to get_application_url() to avoid hardcoded URLs, reducing flakiness on Windows. #55623, #54974, #54924, #54911, #54704, #54903, #54882, #54877, #54631, #53933
• Semaphore bug fix. Corrected race where more workers than allowed could acquire the semaphore. #55147
• LongPollClient cancellation. Prevented spurious cancellation of listen_for_change. #54832
• Backpressure error code. gRPC now returns RESOURCE_EXHAUSTED instead of UNAVAILABLE on overload. #54537
• Logging improvements. Added request IDs to proxy access logs; avoided duplicate shutdown log lines. #54657, #54534
• Test stability. Various waits, deflakes, and sync fixes across Serve tests. #54794, #54522, #54585

📖 Documentation:

• Unexpected queuing behavior. Documented quirks in handle request queuing. #54542

🏗 Architecture refactoring:

• Router/handle internals refactored for clarity and future feature expansion. #55635
• Model composition benchmarks. Added benchmarking to track performance of common composition patterns. #55549
• Constants refactor. Utility functions moved out of constants.py for better readability and stricter env var validation. #54944, #55464
• Ray internals migration. Moved usage, ray_option_utils, and selected constants from _private to _common. #54915, #54578

Ray Serve/Data LLM

🎉 New Features:

• Prefix cache-aware router with PrefixCacheAffinityRouter for optimized cache utilization. (#55218, #55588)
• Reset prefix cache remote method for dynamic cache management. (#55658)
• LMCacheConnectorV1 support for kv_transfer_config to enhance key-value transfer configurations. (#54579)
• LLMServer and LLMEngine major refactor for 100% vLLM serve frontend compatibility. (#54554)

💫 Enhancements:

• vLLM engine upgrade to version 0.10.0 with improved performance and compatibility. (#55067)
• Enhanced error handling for invalid model_id parameters with clearer error messages. (#55589)
• Improved telemetry handling with better race condition management for push operations. (#55558)
• Optimized deployment defaults with better configuration values to prevent bottlenecks. (#54696)
• LoRA workflow improvements with refactored downloading and utility functions. (#54946)
• LLMServer refactor to synchronous initialization for better reliability. (#54835)
• Mistral tokenizer support for tekken tokenizer compatibility. (#54666)
• Smart batching logic that skips batching when batch_interval_ms == 0. (#54751)
• Dashboard enhancements with improved LLM metrics and monitoring capabilities. (#54797)

🔨 Fixes:

• Pyright linting corrections for Ray Serve LLM examples. (#55284)
• Test stability improvements for DeepSeek model and vLLM engine processor tests. (#55401, #55120)
• Serialization fixes for ChatCompletionRequest tool_calls ValidatorIterator objects. (#55538)

📖 Documentation:

• Prefix cache router documentation with comprehensive usage examples. (#55218)
• Multi-LoRA documentation improvements with clearer setup instructions. (#54788)
• STRICT_PACK strategy FAQ documentation explaining data.llm packing behavior. (#55505)

🏗 Architecture refactoring:

• Docker image optimizations with UCX and NCCL updates, plus GKE GPU operator compatibility paths. (#54598, #55206)

RLlib

🎉 New Features:

• Implemented Implicit Q-Learning (IQL). (#55304, #55422)
• DreamerV3 is now available in PyTorch. (#45463, #55140)
• Discrete actions support for SAC. (#53982)

💫 Enhancements:

• Upgraded RLlink protocol for external env/simulator training. (#53550)
• Performance improvements in Offline RL API through switching to iter_torch_batches. (#54277)
• Added an example for curriculum learning in Atari Pong. (#55304)

🔨 Fixes:

• Corrected TensorType handling. (#55694)
• Fixed a bug with multi-learner setups in Offline RL API. (#55693)
• Addressed ImportError in Atari examples. (#54967)
• Fixed some bugs in the docs for IQL and CQL. (#55614)
• Increased default timesteps on two experiments. (#54185)
• Fixed TorchMultiCategorical.to_deterministic when having different number of categories and logits with time dimension. (#54414)
• Added missing documentation for SACConfig's training(). (#53918)
• Fixed bug in restore_from_path such that connector states are also restored on remote EnvRunners. (#54672)
• Fixed missing support for config.count_steps_by = "agent_steps". (#54885)
• Added missing colon to CUBLAS_WORKSPACE_CONFIG. (#53913)
• Removed rllib_contrib completely from RLlib. (#55182)

🏗 Architecture refactoring:

• Deprecated TensorFlow support from new API stack. (#55042)
• Deprecated input/output specs from RLModule. (#55141)
• Deprecated --enable-new-api-stack flag from all scripts. (#54853, #54702)

Ray Core

🎉 New Features:

• [Core] Add AcceleratorManager implementation for Rebellions NPU (#53985)

💫 Enhancements:

• [core][gpu-objects] Garbage collection (#53911)
• [core] Support pip_install_options for pip (#53551)
• [core][gpu-objects] Move data transfers to a background thread (#54256)
• [core][gpu-objects] Pass tensor_transport to store_task_errors even if the actor task throws an exception (#55427)
• [core][gpu-objects] Exception handling for application errors (#55442)
• [core][gpu-object] Add a user-facing call to wait for tensor to be freed (#55076)
• [Core] Bind ray internal servers to the specified node ip instead of 0.0.0.0 which improves the security (#55178, #55210, #55298)
• [core] Fallback unserializable exceptions to their string representation (#55476)

🔨 Fixes:

• [core] Fix objects_valid check failure with except from BaseException (#55602)
• [core][gpu-objects] Avoid triggering a KeyError by the GPU object GC callback for intra-actor communication (#54556)
• [core] fix checking for uv existence during ray_runtime setup (#54141)
• [core][autoscaler][v1] add heartbeat timeout logic to determine node activity status (#54030)
• [core] prevent sending SIGTERM after calling Worker::MarkDead (#54377)
• [Core] Fixed the bug where the head was unable to submit tasks after redis is turned on. (#54267)
• [Core] [Azure] query for supported Microsoft.Network/virtualNetworks API versions instead of relying on resource_client.DEFAULT_API_VERSION (#54874)
• [core] Fix possible race by checking node cache status instead of just subscription (#54745)
• [core] Fix get actor timeout multiplier (#54525)
• [core]: Use a temporary file to share default worker path in runtime env (#53653)
• [core] Fix check fail when task buffer periodical runner runs before RayEvent is initialized (#55249)
• [core] Patch grpc with RAY_num_grpc_threads to control grpc thread count (#54988)
• [core][gpu-objects] Always write to GPUObjectStore to avoid _get_tensor_meta() from hanging indefinitely. (#55433)
• [Core] Core Worker GetObjStatus GRPC Fault Tolerance (#54567)

📖 Documentation:

• Added guide on using type hints with Ray Core. (#55013)

🏗 Architecture refactoring:

• Migrate metric collection from opencensus to opentelemetry (#53098, #53740)

Dashboard

💫 Enhancements:

• Grafana: new Operator filter for Data; Prometheus adds RayNodeType label on for nodes. (#55493, #55192)

🔨 Fixes:

• Removed references to a deleted Data metrics panel. (#55478)

Ray Images

🎉 New Features:

• Added cuda 12.6 based images (#55425, #55444)

💫 Enhancements:

• Upgraded protobuf to v4 (#54496)

Docs

💫 Enhancements:

• KubeRay docs: added InteractiveMode quick-start details; expanded Core type-hints guidance; Serve LLM example coverage; Data LLM batching FAQ (#55570, #55284)

🔨 Fixes:

• Various formatting/mis-highlighting and lints across Train/Tune/Serve LLM docs. (#55284, #54763)

Thanks!

Thank you to everyone who contributed to this release! @pavitrabhalla, @Daraan, @Sparks0219, @daiping8, @abrarsheikh, @sven1977, @Toshaksha, @bveeramani, @MengjinYan, @GokuMohandas, @codope, @nadongjun, @SolitaryThinker, @matthewdeng, @elliot-barn, @isimluk, @avibasnet31, @OneSizeFitsQuorum, @Future-Outlier, @marosset, @jackfrancis, @kshanmol, @eicherseiji, @dayshah, @iamjustinhsu, @Qiaolin-Yu, @goutamvenkat-anyscale, @Yicheng-Lu-llll, @yantarou, @rclough, @zcin, @NeilGirdhar, @VarunBhandary, @400Ping, @akshay-anyscale, @vickytsang, @xushiyan, @JasonLi1909, @n-elia, @simonsays1980, @dragongu, @Kishanthan, @ruisearch42, @jectpro7, @TimothySeah, @liulehui, @rueian, @HollowMan6, @akyang-anyscale, @axreldable, @czgdp1807, @alanwguo, @justinvyu, @ok-scale, @my-vegetable-has-exploded, @landscapepainter, @fscnick, @machichima, @mpashkovskii, @ZacAttack, @gvspraveen, @sword865, @lmsh7, @Ziy1-Tan, @rebel-scottlee, @sampan-s-nayak, @coqian, @can-anyscale, @Bye-legumes, @win5923, @MortalHappiness, @angelinalg, @khluu, @aslonnie, @krishnakalyan3, @minosvasilias, @x-tong, @xinyuangui2, @raulchen, @Yangruipis, @edoakes, @kevin85421, @wingkitlee0, @Fokko, @cristianjd, @srinathk10, @owenowenisme, @JoshKarpel, @MengqingCao, @leopardracer, @westonpace, @LeslieWongCV, @VassilisVassiliadis, @crypdick, @alexeykudinkin, @mjacar, @kunling-anyscale, @saihaj, @kouroshHakha, @ema-pe, @markjm, @avigyabb, @dshepelev15, @mauvilsa, @omatthew98, @nrghosh, @ryanaoleary, @Aydin-ab, @lk-chen, @stephanie-wang, @harshit-anyscale, @jjyao, @bullgom, @Yevet, @israbbani

Release Highlights

• Ray Data: This release features a new Delta Lake and Unity Catalog integration and performance improvements to various reading/writing operators.
• Ray Core: Enhanced GPU object support with intra-process communication and improved Autoscaler v2 functionality
• Ray Train: Improved hardware metrics integration with Grafana and enhanced collective operations support
• Ray Serve LLM: This release features early proof of concept for prefill-decode disaggregation deployment and LLM-aware request routing such as prefix-cache aware routing.
• Ray Data LLM: Improved throughput and CPU memory utilization for ray data workers.

Ray Libraries

Ray Data

🎉 New Features:

• Add reading from Delta Lake tables and Unity Catalog integration (#53701)
• Enhanced pin_memory support in iter_torch_batches (#53792)
• Add pin_memory to iter_torch_batches (#53792)

💫 Enhancements:

• Re-enabled sorting in Ray Data tests with performance improvements (#54475)
• Enhanced handling of mismatched columns and pandas.NA values (#53861, #53859)
• Improved read_text trailing newline semantics (#53860)
• Optimized backpressure handling with policy-based resource management (#54376)
• Enhanced write_parquet with support for both partition_by and row limits (#53930)
• Prevent filename collisions on write operations (#53890)
• Improved execution performance for One Hot encoding in preprocessors (#54022)

🔨 Fixes:

• Fixing map_groups issues (#54462)
• Prevented Op fusion for streaming repartition to avoid performance degradation (#54469)
• Fixed ActorPool autoscaler scaling up logic (#53983)
• Resolved empty dataset repartitioning issues (#54107)
• Fixed PyArrow overflow handling in data processing (#53971, #54390)
• Fixed IcebergDatasink to properly generate individual file uuids (#52956)
• Avoid OOMs with read_json(..., lines=True) (#54436)
• Handle HuggingFace parquet dataset resolve URLs (#54146)
• Fixed BlockMetadata derivation for Read operator (#53908)

📖 Documentation:

• Updated AggregateFnV2 documentation to clarify finalize method (#53835)
• Improved preprocessor and vectorizer API documentation

Ray Train

🎉 New Features:

• Added broadcast_from_rank_zero and barrier collective operations (#54066)
• Enhanced hardware metrics integration with Grafana dashboards (#53218)
• Added support for dynamically loading callbacks via environment variables (#54233)

💫 Enhancements:

• Improved checkpoint population from before_init_train_context (#54453)
• Enhanced controller state logging and metrics (#52805)
• Added structured logging environment variable support (#52952)
• Improved handling of Noop scaling decisions for smoother scaling logic (#53180)
• Logging of controller state transitions to aid in debugging and analysis (#53344)

🔨 Fixes:

• Fixed GPU tensor reporting in ray.train.report (#53725)
• Enhanced move_tensors_to_device utility for complex tensor structures (#53109)
• Improved worker health check error handling with trace information (#53626)
• Fixed GPU transfer support for non-contiguous tensors (#52548)
• Force abort on SIGINT spam and do not abort finished runs (#54188)

📖 Documentation:

• Updated beginner PyTorch example (#54124)
• Added documentation for ray.train.collective APIs (#54340)
• Added a note about PyTorch DataLoader's multiprocessing and forkserver usage (#52924)
• Fixed various docstring format and indentation issues (#52855, #52878)
• Added note that ray.train.report API docs should mention optional checkpoint_dir_name (#54391)

🏗 Architecture refactoring:

• Removed subclass relationship between RunConfig and RunConfigV1 (#54293)
• Enhanced error handling for finished training runs (#54188)
• Deduplicated ML doctest runners in CI for efficiency (#53157)
• Converted isort configuration to Ruff for consistency (#52869)

Ray Tune

💫 Enhancements:

• Updated test_train_v2_integration to use the correct RunConfig (#52882)

🔨 Fixes:

• Fixed RayTaskError serialization logic (#54396)
• Improved experiment restore timeout handling (#53387)

📖 Documentation:

• Replaced session.report with tune.report and corrected import paths (#52801)
• Removed outdated graphics cards reference in docs (#52922)
• Fixed various docstring format issues (#52879)

Ray Serve

🎉 New Features:

• Added RouterConfig field to DeploymentConfig for custom RequestRouter configuration (#53870)
• Added support for implementing custom request routing algorithms (#53251)

💫 Enhancements:

• Enhanced FastAPI ingress deployment validation for multiple deployments (#53647)
• Optimized get_live_deployments performance (#54454)
• Progress towards making ray.serve.llm compatible with vLLM serve frontend (#54481, #54443, #54440)

🔨 Fixes:

• Fixed deployment scheduler issues with component scheduling (#54479)
• Fixed runtime_env validation for py_modules (#53186)
• Added descriptive error message when deployment name is not found (#45181)

📖 Documentation:

• Added troubleshooting guide for DeepSeek/multi-node GPU deployment on KubeRay (#54229)
• Updated the guide on serving models with Triton Server in Ray Serve
• Added documentation for custom request routing algorithms
• Added custom request router docs (#53511)

🏗 Architecture refactoring:

• Remove indirection layers of node initialization (#54481)
• Incremental refactor of LLMEngine (#54443)
• Remove random v0 logic from serve endpoints (#54440)
• Remove usage of internal_api.memory_summary() (#54417)
• Remove usage of ray._private.state (#54140)

Ray Serve/Data LLM

🎉 New Features

• Support separate deployment config for PDProxy in PrefixAwareReplicaSet (#53935)
• Support for prefix-aware request router (#52725)

💫 Enhancements

• Log engine stats after each batch task is done. (#54360)
• Decouple max_tasks_in_flight from max_concurrent_batches (#54362)
• Make llm serve endpoints compatible with vLLM serve frontend, including streaming, tool_code, and health check support (#54440)
• Remove botocore dependency in Ray Serve LLM (#54156)
• Update vLLM version to 0.9.2 (#54407)

🔨 Fixes

• Fix health check in prefill disagg (#53937)
• Fix doc to only support int concurrency (#54196)
• Fix vLLM batch test by changing to Pixtral (#53744)
• Fix pickle error with remote code models in vLLM Ray workloads (#53868)
• Adaption of the change of vllm.PoolingOutput (#54467)

📖 Documentation

• Ray serve/lora doc fix (#53553)
• Add Ray serve/LLM doc (#52832)
• Add a doc snippet to inform users about existing diffs between vLLM and Ray Serve LLM behavior in some APIs like streaming, tool_code, and health check (#54123)
• Troubleshooting DeepSeek/multi-node GPU deployment on KubeRay (#54229)

🏗 Architecture refactoring

• Make llm serve endpoints compatible with vLLM serve frontend, including streaming, tool_code, and health check support (#54490)
• Prefix-aware scheduler [2/N] Configure PrefixAwareReplicaSet to correctly handle the number of available GPUs for each worker and to ensure efficient GPU utilization in vLLM (#53192)
• Organize spread out utils.py (#53722)
• Remove ImageRetriever class and related tests from the LLM serving codebase. (#54018)
• Return a batch of rows in the udf instead of row by row (#54329)

RLlib

🎉 New Features:

• Implemented Offline Policy Evaluation (OPE) via Importance Sampling (#53702)
• Enhanced ConnectorV2 ObservationPreprocessor APIs with multi-agent support (#54209)
• Add GPU inference to offline evaluation (#52718)

💫 Enhancements:

• Enhanced MetricsLogger to handle tensors in state management (#53514)
• Improved env seeding in EnvRunners with deterministic training example rewrite (#54039)
• Cleanup of meta learning classes and examples (#52680)

🔨 Fixes:

• Fixed EnvRunner restoration when no local EnvRunner is available (#54091)
• Fixed shapes in explained_variance for recurrent policies (#54005)
• Resolved device check issues in Learner implementation (#53706)
• Enhanced numerical stability in MeanStdFilter (#53484)
• Fixed weight synching in offline evaluation (#52757)
• Fixed bug in split_and_zero_pad utility function (#52818)

📖 Documentation:

• Do-over of examples for connector pipelines (#52604)
• Remove "new API stack" banner from all RLlib docs pages as it's now the default (#54282)

Ray Core

🎉 New Features:

• Enhanced GPU object support with intra-process communication (#53798)
• Integrated single-controller collective APIs with GPU objects (#53720)
• Added support for ray.get on driver process for GPU objects (#53902)
• Supporting allreduce on list of input nodes in compiled graphs (#51047)
• Add single-controller API for ray.util.collective and torch gloo backend (#53319)

💫 Enhancements:

• Improved autoscaler v2 functionality with cloud instance ID reusing (#54397)
• Enhanced cluster task manager with better resource management (#54413)
• Upgraded OpenTelemetry SDK for better observability (#53745)
• Improved actor scheduling to prevent deadlocks in ordered actors (#54034)
• Enhanced get_max_resources_from_cluster_config functionality (#54455)
• Use std::move in cluster task manager constructor (#54413)
• Improve status messages and add comments about stale seq_no handling (#54470)
• uv run integration is now enabled by default (#53060)

🔨 Fixes:

• Fixed race conditions in object eviction and repinning for recovery (#53934)
• Resolved GCS crash issues on duplicate MarkJobFinished RPCs (#53951)
• Enhanced actor restart handling on node failures (#54088)
• Improved reference counting during worker graceful shutdown (#53002)
• Fix race condition when canceling task that hasn't started yet (#52703)
• Fix the issue where a valid RestartActor rpc is ignored (#53330)
• Fixed "Check failed: it->second.num_retries_left == -1" error (#54116)
• Fix detached actor being unexpectedly killed (#53562)

📖 Documentation:

• Enhanced troubleshooting guides and API documentation
• Updated reStructuredText formatting on Resources page (#53882)
• Fix working code snippets (#52748)
• Add doc for running KubeRay dashboard (#53830)
• Add antipattern for nested ray.get (#43184)

🏗 Architecture refactoring:

• Delete old skipped tests and unused code (#54427)
• Consolidate TaskManager interface (#54317)
• Move dependencies of NodeManager to main.cc for better testability (#53782)
• Use smart pointer in logging.cc (#54351)
• Delete event_label and unused environment variables (#54378, #54095)
• Remove actor task path in normal task submitter (#53996)
• Rename GcsFunctionManager and use fake in test (#53973)

Dashboard

🎉 New Features:

• Add dynolog for on-demand GPU profiling for Torch training (#53191)

💫 Enhancements:

• Added TPU usage metrics to reporter agent (#53678)
• Enhanced GPU profiling manager IP address retrieval (#53807)
• Improved configurability of Grafana dashboard parameters (#53236)
• Add configurability of 'orgId' param for requesting Grafana dashboards (#53236)

🔨 Fixes:

• Fixed Grafana dashboard dropdowns for data and train dashboards (#52752)
• Resolved daylight savings time issues in dashboard (#52755)
• Fix retrieving IP address from the GPUProfilingManager on the dashboard agent (#53807)

Docs 🎉 New Features:

• New end-to-end examples:
  • Multi-modal AI pipeline (#52342)
  • Xgboost tutorial (#52383)
  • Audio transcription and LLM as judge curation (#53189)
  • LLM training and inference (#53415)
  • Scalable video processing (#50965)

💫 Enhancements:

• Add pydoclint to pre-commit (#52974)
• Add vale to pre-commit (#53564)

Breaking Changes

• Removed deprecated ray.workflow package (#53612)
• Removed deprecated storage parameter from ray.init (#53669)
• Removed deprecated ray start CLI options (#53675)
• Removed experimental "array" library (#54105)
• Remove dask from byod 3.9 deps (#54521)

Dependencies & Build

• Added uv binary v0.7.19 for improved package management (#54437)
• Upgraded datasets in release tests (#54425)
• Enhanced wheel building process with single bazel call optimization (#54476)
• Fixed uv run parser for handling extra arguments (#54488)
• Upgrade h11, requests, starlette, jinja2, pyopenssl and cryptography
• Generate multi-arch image indexes (#52816)

Thanks!

Thank you to everyone who contributed to this release! @kouroshHakha, @davidwagnerkc, @MengjinYan, @minerharry, @simonsays1980, @Myasuka, @noemotiovon, @goutamvenkat-anyscale, @harshit-anyscale, @jugalshah291, @tianyi-ge, @sven1977, @crypdick, @JohnsonKuan, @lk-chen, @richardsliu, @alexeykudinkin, @EagleLo, @soffer-anyscale, @zcin, @AdrienVannson, @nilsmelchert, @raulchen, @jujipotle, @DrehanM, @vigneshka, @Ziy1-Tan, @Blaze-DSP, @ArthurBook, @GokuMohandas, @walkoss, @bveeramani, @edoakes, @omatthew98, @SeanQuant, @CheyuWu, @cszhu, @win5923, @kevin85421, @angelinalg, @iamjustinhsu, @eicherseiji, @kunling-anyscale, @vickytsang, @MortalHappiness, @aslonnie, @psr-ai, @sbhat98, @anyadontfly, @marwan116, @cristianjd, @2niuhe, @codope, @fscnick, @ryanaoleary, @srinathk10, @TimothySeah, @han-steve, @Future-Outlier, @Syulin7, @Qiaolin-Yu, @elliot-barn, @JoshKarpel, @dayshah, @can-anyscale, @ok-scale, @mattip, @SolitaryThinker, @owenowenisme, @nehiljain, @GeneDer, @rnkrtt, @israbbani, @DriverSong, @sinalallsite, @pcmoritz, @akyang-anyscale, @xinyuangui2, @nrghosh, @davidxia, @rueian, @stephanie-wang, @jjyao, @chris-ray-zhang, @czgdp1807, @justinvyu, @Daraan, @landscapepainter, @troychiu, @khluu, @hipudding, @ruisearch42, @robertnishihara, @ArturNiederfahrenhorst, @abrarsheikh, @alanwguo, @HollowMan6, @ran1995data, @matthewdeng

Ray 2.47.1 fixed an issue where Ray failed to start on Mac (https://github.com/ray-project/ray/pull/53807)

Release Highlights

• Prefill disaggregation is now supported in initial support in Ray Serve LLM (#53092). This is critical for production LLM serving use cases.
• Ray Data features a variety of performance improvements (locality-based scheduling, non-blocking execution) as well as improvements to observability, preprocessors, and other stability fixes.
• Ray Serve now features custom request routing algorithms, which is critical for high throughput traffic for large model use cases.

Ray Libraries

Ray Data

🎉 New Features:

• Add save modes support to file data sinks (#52900)
• Added flattening capability to the Concatenator preprocessor to support output vectorization use cases (#53378)

💫 Enhancements:

• Re-enable Actor locality-based scheduling. This PR also improves algorithms for ranking the locations for the bundle. (#52861)
• Disable blocking pipeline by default until Actor Pool fully scales up to min actors (#52754)
• Progress bar and dashboard improvements to show name of partial functions properly(#52280)

🔨 Fixes:

• Make Ray Data from_torch respect Dataset len (#52804)
• Fixing flaky aggregation test (#53383)
• Fix race condition bug in fault tolerance by disabling on_exit hook (#53249)
• Fix move_tensors_to_device utility for the list/tuple[tensor] case (#53109)
• Fix ActorPool scaling to avoid scaling down when the input queue is empty (#53009)
• Fix internal queues accounting for all Operators w/ an internal queue (#52806)
• Fix backpressure for FileBasedDatasource. This fixes potential OOMs for workloads using FileBasedDatasources (#52852)

📖 Documentation:

• Fix working code snippets (#52748)
• Improve AggregateFnV2 docstrings and examples (#52911)
• Improved documentation for vectorizers and API visibility in Data (#52456)

Ray Train

🎉 New Features:

• Added support for configuring Ray Train worker actor runtime environments. (#52421)
• Included Grafana panel data in Ray Train export for improved monitoring. (#53072)
• Introduced a structured logging environment variable to standardize log formats. (#52952)
• Added metrics for TrainControllerState to enhance observability. (#52805)

💫 Enhancements:

• Logging of controller state transitions to aid in debugging and analysis. (#53344)
• Improved handling of Noop scaling decisions for smoother scaling logic. (#53180)

🔨 Fixes:

• Improved move_tensors_to_device utility to correctly handle list / tuple of tensors. (#53109)
• Fixed GPU transfer support for non-contiguous tensors. (#52548)
• Increased timeout in test_torch_device_manager to reduce flakiness. (#52917)

📖 Documentation:

• Added a note about PyTorch DataLoader’s multiprocessing and forkserver usage. (#52924)
• Fixed various docstring format and indentation issues. (#52855, #52878)
• Removed unused "configuration-overview" documentation page. (#52912)
• General typo corrections. (#53048)

🏗 Architecture refactoring:

• Deduplicated ML doctest runners in CI for efficiency. (#53157)
• Converted isort configuration to Ruff for consistency. (#52869)
• Removed unused PARALLEL_CI blocks and combined imports. (#53087, #52742)

Ray Tune

💫 Enhancements:

• Updated test_train_v2_integration to use the correct RunConfig. (#52882)

📖 Documentation:

• Replaced session.report with tune.report and corrected import paths. (#52801)
• Removed outdated graphics cards reference in docs. (#52922)
• Fixed various docstring format issues. (#52879)

Ray Serve

🎉 New Features:

• Added support for implementing custom request routing algorithms. (#53251)
• Introduced an environment variable to prioritize custom resources during deployment scheduling. (#51978)

💫 Enhancements:

• The ingress API now accepts a builder function in addition to an ASGI app object. (#52892)

🔨 Fixes:

• Fixed runtime_env validation for py_modules. (#53186)
• Disallowed special characters in Serve deployment and application names. (#52702)
• Added a descriptive error message when a deployment name is not found. (#45181)

📖 Documentation:

• Updated the guide on serving models with Triton Server in Ray Serve.
• Added documentation for custom request routing algorithms.

Ray Serve/Data LLM

🎉 New Features:

• Added initial support for prefill decode disaggregation (#53092)
• Expose vLLM Metrics to serve.llm API (#52719)
• Embedding API (#52229)

💫 Enhancements:

• Allow setting name_prefix in build_llm_deployment (#53316)
• Minor bug fix for 53144: stop tokens cannot be null (#53288)
• Add missing repetition_penalty vLLM sampling parameter (#53222)
• Mitigate the serve.llm streaming overhead by properly batching stream chunks (#52766)
• Fix test_batch_vllm leaking resources by using larger wait_for_min_actors_s

🔨 Fixes:

• LLMRouter.check_health() should check LLMServer.check_health() (#53358)
• Fix runtime passthrough and auto-executor class selection (#53253)
• Update check_health return type (#53114)
• Bug fix for duplication of <bos> token (#52853)
• In stream batching, first part of the stream was always consumed and not streamed back from the router (#52848)

RLlib

🎉 New Features:

• Add GPU inference to offline evaluation. (#52718)

💫 Enhancements:

• Do-over of examples for connector pipelines. (#52604)
• Cleanup of meta learning classes and examples. (#52680)

🔨 Fixes:

• Fixed weight synching in offline evaluation. (#52757)
• Fixed bug in split_and_zero_pad utility function (related to complex structures vs simple values or np.arrays). (#52818)

Ray Core

💫 Enhancements:

• uv run integration is now enabled by default, so you don't need to set the RAY_RUNTIME_ENV_HOOK any more (#53060). If you rely on the previous behavior where uv run only runs the Ray driver but not the workers in the uv environment, you can switch back to the old behavior by setting the RAY_ENABLE_UV_RUN_RUNTIME_ENV=0 environment variable.
• Record gcs process metrics (#53171)

🔨 Fixes:

• Improvements for using RuntimeEnv in the Job Submission API. (#52704)
• Close unused pipe file descriptor of child processes of Raylet (#52700)
• Fix race condition when canceling task that hasn't started yet (#52703)
• Implement a thread pool and call the CPython API on all threads within the same concurrency group (#52575)
• cgraph: Fix execution schedules with collective operations (#53007)
• cgraph: Fix scalar tensor serialization edge case with serialize_to_numpy_or_scalar (#53160)
• Fix the issue where a valid RestartActor rpc is ignored (#53330)
• Fix reference counter crashes during worker graceful shutdown (#53002)

Dashboard

🎉 New Features:

• train: Add dynolog for on-demand GPU profiling for Torch training (#53191)

💫 Enhancements:

• Add configurability of 'orgId' param for requesting Grafana dashboards (#53236)

🔨 Fixes:

• Fix Grafana dashboards dropdowns for data and train dashboard (#52752)
• Fix dashboard for daylight savings (#52755)

Ray Container Images

💫 Enhancements:

• Upgrade h11 (#53361), requests, starlette, jinja2 (#52951), pyopenssl and cryptography (#52941)
• Generate multi-arch image indexes (#52816)

Docs

🎉 New Features:

• End-to-end example: Entity recognition with LLMs (#52342) - new end-to-end example
• End-to-end example: xgboost tutorial (#52383)
• End-to-end tutorial for audio transcription and LLM as judge curation (#53189)

💫 Enhancements:

• Adds pydoclint to pre-commit (#52974)

Thanks!

Thank you to everyone who contributed to this release!

@NeilGirdhar, @ok-scale, @JiangJiaWei1103, @brandonscript, @eicherseiji, @ktyxx, @MichalPitr, @GeneDer, @rueian, @khluu, @bveeramani, @ArturNiederfahrenhorst, @c8ef, @lk-chen, @alanwguo, @simonsays1980, @codope, @ArthurBook, @kouroshHakha, @Yicheng-Lu-llll, @jujipotle, @aslonnie, @justinvyu, @machichima, @pcmoritz, @saihaj, @wingkitlee0, @omatthew98, @can-anyscale, @nadongjun, @chris-ray-zhang, @dizer-ti, @matthewdeng, @ryanaoleary, @janimo, @crypdick, @srinathk10, @cszhu, @TimothySeah, @iamjustinhsu, @mimiliaogo, @angelinalg, @gvspraveen, @kevin85421, @jjyao, @elliot-barn, @xingyu-long, @LeoLiao123, @thomasdesr, @ishaan-mehta, @noemotiovon, @hipudding, @davidxia, @omahs, @MengjinYan, @dengwxn, @MortalHappiness, @alhparsa, @emmanuel-ferdman, @alexeykudinkin, @KunWuLuan, @dev-goyal, @sven1977, @akyang-anyscale, @GokuMohandas, @raulchen, @abrarsheikh, @edoakes, @JoshKarpel, @bhmiller, @seanlaii, @ruisearch42, @dayshah, @Bye-legumes, @petern48, @richardliaw, @rclough, @israbbani, @jiwq

Release Highlights

The 2.46 Ray release comes with a couple core highlights:

1. Ray Data now supports hash shuffling for repartition and aggregations, along with support for joins. This enables many new data processing workloads to be run on Ray Data. Please give it a try and let us know if you have any feedback!
2. Ray Serve LLM now supports vLLM v1 to be forward-compatible with upcoming vLLM releases. This also opens up significant performance improvements that come with vLLM's v1 refactor.
3. There is a new Train Grafana dashboard which provides in-depth metrics on Grafana for better metrics on training workloads.

Ray Libraries

Ray Data

🎉 New Features:

• Adding support for hash-shuffle based repartitioning and aggregations (#52664)
• Added support for Joins (using hash-shuffle) (#52728)
• [LLM] vLLM support upgrades to 0.8.5 (#52344)

💫 Enhancements:

• Add memory attribute to ExecutionResources (#51127)
• Support ray_remote_args for read_tfrecords #52450
• [data.dashboard] Skip reporting internal metrics (#52666)
• Add PhysicalOperator.min_max_resource_usage_bounds (#52502)
• Speed up printing the schema (#52612)
• [data.dashboard] Dataset logger for worker (#52706)
• Support new pyiceberg version (#51744)
• Support num_cpus, memory, concurrency, batch_size for preprocess (#52574)

🔨 Fixes:

• Handle Arrow Array null types in to_numpy (#52572)
• Fix S3 serialization wrapper compatibility with RetryingPyFileSystem (#52568)
• Fixing Optimizer to apply rules until plan stabilize; (#52663)
• Fixing FuseOperators rule to properly handle the case of transformations drastically changing size of the dataset (#52570)

📖 Documentation:

• [LLM] Improve concurrency settings, improve prompt to achieve better throughput (#52634)

Ray Train

🎉 New Features:

• Add initial Train Grafana dashboard (#52709)

💫 Enhancements:

• Lazily import torch FSDP for ray.train.torch module to improve performance and reduce unnecessary dependencies (#52707)
• Deserialize the user-defined training function directly on workers, improving efficiency (#52684)

🔨 Fixes:

• Fixed error when no arguments are passed into TorchTrainer (#52693)

📖 Documentation:

• Added new XGBoostTrainer user guide (#52355)

🏗 Architecture refactoring:

• Re-enabled isort for python/ray/train to maintain code formatting consistency (#52717)

Ray Tune

📖 Documentation:

• Fixed typo in Ray Tune PyTorch Lightning docs (#52756)

Ray Serve

💫 Enhancements:

• [LLM] Refactor LLMServer and LLMEngine to not diverge too much from vllm chat formatting logic (#52597)
• Bump vllm from 0.8.2 to 0.8.5 in /python (#52344)
• [LLM] Add router replicas and batch size to llm config (#52655)

🔨 Fixes:

• Request cancellation not propagating correctly across deployments (#52591)
• BackpressureError not properly propagated in FastAPI ingress deployments (#52397)
• Hanging issue when awaiting deployment responses (#52561)
• [Serve.llm] made Ray Serve LLM compatible with vLLM v1 (#52668)

📖 Documentation:

• [Serve][LLM] Add doc for deploying DeepSeek (#52592)

RLLib

🎉 New Features:

• Offline Evaluation with loss function for Offline RL pipeline. Introduces three new callbacks, on_offline_evaluate_start, on_offline_evaluate_end, on_offline_eval_runners_recreated (#52308)

💫 Enhancements:

• New custom_data attribute for SingleAgentEpisode and MultiAgentEpisode to store custom metrics. Deprecates add|get_temporary_timestep_data() (#52603)

Ray Core

💫 Enhancements:

• Only get serialization context once for all .remote args (#52690)
• Add grpc server success and fail count metric (#52711)

🔨 Fixes:

• Fix open leak for plasma store memory (shm/fallback) by workers (#52622)
• Assure closing of unused pipe for dashboard subprocesses (#52678)
• Expand protection against dead processes in reporter agent (#52657)
• [cgraph] Separate metadata and data in cross-node shared memory transport (#52619)
• Fix JobID check for detached actor tasks (#52405)
• Fix potential log loss of tail_job_logs (#44709)

🏗 Architecture refactoring:

• Cancel tasks when an owner dies instead of checking if an owner is dead during scheduling (#52516)
• Unify GcsAioClient and GcsClient (#52735)
• Remove worker context dependency from the task receiver (#52740)

Dashboard

🎉 New Features:

• Ray Train Grafana Dashboard added with a few built-in metrics. More to come.

Thanks!

Thank you to everyone who contributed to this release! @kevin85421, @edoakes, @wingkitlee0, @alexeykudinkin, @chris-ray-zhang, @sophie0730, @zcin, @raulchen, @matthewdeng, @abrarsheikh, @popojk, @Jay-ju, @ruisearch42, @eicherseiji, @lk-chen, @justinvyu, @dayshah, @kouroshHakha, @NeilGirdhar, @omatthew98, @ishaan-mehta, @davidxia, @ArthurBook, @GeneDer, @srinathk10, @dependabot[bot], @JoshKarpel, @aslonnie, @khluu, @can-anyscale, @israbbani, @saihaj, @MortalHappiness, @alanwguo, @bveeramani, @iamjustinhsu, @Ziy1-Tan, @xingyu-long, @simonsays1980, @fscnick, @chuang0221, @sven1977, @jjyao

Ray Core

💫 Enhancements

• Make Object Store Fallback Directory configurable (#51189).
• [cgraph] Support with_tensor_transport(transport='shm') (#51872).
• [cgraph] Support reduce scatter and all gather collective for GPU communicator in compiled graph (#50624).

🔨 Fixes

• Make sure KillActor RPC with force_kill=True can actually kill the threaded actor (#51414).
• [Autoscaler] Do not remove idle nodes for upcoming placement groups (#51122).
• Threaded actors get stuck forever if they receive two exit signals (#51582).
• [cgraph] Fix illegal memory access of cgraph when used in PP (#51734).
• Avoid resubmitted actor tasks from hanging indefinitely (#51904).
• Fix interleaved placement group creation process due to node failure (#52202).
• Flush task events in CoreWorker::Shutdown instead of CoreWorker::Disconnect (#52374).

🏗 Architecture refactoring

• Split dashboard single process into multiple processes to improve stability and avoid interference between different heads (#51282, #51489, #51555, #51507, #51587, #51553, #51676, #51733, #51809, #51877, #51876, #51980, #52114).

Ray Libraries

Ray Data

🎉 New Features

• New ClickHouse sink via Dataset.write_clickhouse() (#50377)
• Support ray_remote_args_fn in Dataset.groupby().map_groups() to set per-group runtime env and resource hints (#51236)
• Expose Dataset.name / set_name as a public API for easier lineage tracking (#51076)
• Allow async callable classes in Dataset.flat_map() (#51180)
• Introduce Ruleset abstraction for rule-based query optimisation (#51558)
• Add seamless conversion from Daft DataFrame to Ray Dataset (#51531)
• Improved support for line-delimited JSONL reading in read_json() (#52083)
• Provide Dataset.export_metadata() for schema & stats snapshots (#52227)

💫 Enhancements

• Improved performance of sorting and sort-shuffle based operations (by more than 5x in benchmarks) (#51943)
• Metrics: number of map-actor workers alive / pending / restarting (#51082)
• Continuous memory-usage polling per map task (#51324)
• Auto-tune map-task memory based on output size (#51536)
• More informative back-pressure progress bar (#51697)
• Faster RefBundle.get_cached_location() lookup (#52097)
• Speed-up for PandasBlock.size_bytes() (#52510)
• Expanded BlockColumnAccessor utilities and ops (#51326, #51571)

🔨 Fixes

• Correct MapTransformFn.__eq__ equality check (#51434)
• Persist unresolved wildcard paths in FileBasedDataSource (#51424)
• Repair Hugging Face dynamic-module loading on workers (#51488)
• Prevent HTTP URLs from being expanded by _expand_paths (#50178)
• Fix Databricks host-URL parsing in Delta datasource (#49926)
• Restore reproducibility of Dataset.random_sample() (#51401)
• Correct RandomAccessDataset.multiget() return values (#51421)
• Ensure executor shutdown after schema fetch to avoid leaked actors (#52379)
• Repair streaming shutdown regression (#52509)
• Honour minimum resource reservation in ResourceManager (#52226)

📖 Documentation

• Clarified shuffle-section wording (#51289)
• Documented concurrency semantics in API reference (#51963)
• Updated Ray Data guides for the 2.45 release (#52082)

Ray Train

🎉 New Features

• Fold v2.LightGBMTrainer API into the public trainer class as an alternate constructor (#51265).

💫 Enhancements

• Use the user-defined function name as the training thread name (#52514).
• Upgrade LightGBM to version 4.6.0 (#52410).
• Adjust test size further for better results (#52283).
• Log errors raised by workers during training (#52223).
• Add worker group setup finished log to track progress (#52120).
• Change test_telemetry to medium size (#52178).
• Improve dataset name observability for better tracking (#52059).
• Differentiate between train v1 and v2 export data for clarity (#51728).
• Include scheduling status detail to improve debugging (#51480).
• Move train library usage check to Trainer initialization (#50966).

🔨 Fixes

• Separate OutputSplitter._locality_hints from actor_locality_enabled and locality_with_output (#52005).
• Fix print redirection to handle new lines correctly (#51542).
• Mark RunAttempt workers as dead after completion to avoid stale states (#51540).
• Fix setup_wandb rank_zero_only logic (#52381).

📖 Documentation

• Add links to the Train v2 migration guide in the Train API pages (#51924).

🏗 Architecture refactoring

• Replace AMD device environment variable with HIP_VISIBLE_DEVICES (#51104).
• Remove unnecessary string literal splits (#47360).

Ray Tune

📖 Documentation

• Improve Tune documentation structure (#51684).
• Fix syntax errors in Ray Tune example pbt_ppo_example.ipynb (#51626).

Ray Serve

🎉 New Features

• Add request timeout sec for gRPC (#52276).
• [Serve.llm] ray.llm support custom accelerators (#51359).

💫 Enhancements

• Improve Serve deploy ignore behavior (#49336).
• [Serve.llm] Telemetry GPU type fallback to cluster hardware when unspecified (#52003).

🔨 Fixes

• Fix multiplex fallback logic during burst requests (#51389).
• Don't stop retrying replicas when a deployment is scaling back up from zero (#51600).
• Remove RAY_SERVE_ENABLE_QUEUE_LENGTH_CACHE flag (#51649).
• Remove RAY_SERVE_EAGERLY_START_REPLACEMENT_REPLICAS flag (#51722).
• Unify request cancellation errors (#51768).
• Catch timeout error when checking if proxy is dead (#52002).
• Suppress cancelled errors in proxy (#52423).
• [Serve.llm] Fix loading model from remote storage and add docs (#51617).
• [Serve.llm] Fix ServeReplica deployment failure for DeepSeek (#51989).
• [Serve.llm] Check GPUType enum value rather than enum itself (#52037).
• [Serve.llm] Fix inconsistent v0/v1 config passed to vLLM (#52185).
• [Serve.llm] Fix V0/V1 auto-resolution more broadly (#52357).
• [Serve.llm] Fix getting prompt limit for input-too-long error (#52315).

📖 Documentation

• Update deployment status docs (#51610).
• Update engine kwargs in LoRA instructions (#52203).
• Remove outdated docs about request retries (#52438).
• [Serve.llm] Add gen config related doc (#51572).
• [Serve.llm] Fix loading model from remote storage and add docs (#51617).
• [Serve.llm] Update Ray Serve LLM docs (#51867).
• [Serve.llm] Update model_source doc (#52054).
• [Serve.llm] Add doc example on using tokenizer_pool_size (#52356).
• [Serve.llm] Add API links (#52384).

RLlib

🎉 New Features

• Meta-learning, differentiable learner (#51859, #52494).

💫 Enhancements

• Offline RL:
  • Add configuration parameter ignore_final_observation and automatic episode ID generation (#52107).
  • BC performance improvements and adjustments to new Learner.update logic (#51425).
• Better handling of tensors in MetricsLogger and Stats (#52334).
• Make min/max env steps per evaluation sample call configurable for duration="auto" (#51637).
• Access to all episode chunks in EnvRunner callback on_episode_end (#52252).

🔨 Fixes

• Various smaller fixes (#52503, #51371, #51816, #51621, #44700, #51522, #51406).

🏗 Architecture refactoring

• Remove local EnvRunner (optional) (#52091).

Dashboard

💫 Enhancements

• [Core/Dashboard] Add CPU and memory metrics for subprocesses (#52208).

🔨 Fixes

• [Observability] Fix metrics in the core Grafana dashboard to sum by labels (#51894).
• [Dashboard] Remove millisecond padding in DurationText component (#52395).
• [Core][Dashboard-Agent] Surface the root cause of job submission failure when the dashboard agent fails to launch HTTP server (#51869).

Many thanks to all those who contributed to this release! @samhallam-reverb, @vickytsang, @anyadontfly, @zhaoch23, @bryant1410, @khluu, @akyang-anyscale, @angelinalg, @RocketRider, @wingkitlee0, @robertnishihara, @liuxsh9, @KepingYan, @SumanthRH, @emmanuel-ferdman, @ashwinsnambiar, @ArturNiederfahrenhorst, @KPCOFGS, @Bye-legumes, @400Ping, @dayshah, @aslonnie, @justinvyu, @rugggg, @zhiqiwangebay, @comaniac, @thusoy, @JDarDagran, @chuang0221, @davidxia, @tnixon, @israbbani, @win5923, @leibovitzgil, @simonsays1980, @machichima, @VamshikShetty, @zcin, @lk-chen, @abrarsheikh, @edoakes, @alexeykudinkin, @ruisearch42, @tespent, @jecsand838, @sijieamoy, @can-anyscale, @JonDum, @jyakaranda, @nadongjun, @d-miketa, @MortalHappiness, @kevin85421, @Ziy1-Tan, @matthewdeng, @crypdick, @hongpeng-guo, @richardliaw, @Qiaolin-Yu, @bhmiller, @soffer-anyscale, @kenchung285, @nishi-t, @Drice1999, @ryanaoleary, @chris-ray-zhang, @MengjinYan, @saihaj, @jjyao, @jaganmolleti7, @iamjustinhsu, @fscnick, @pcmoritz, @Jay-ju, @westonpace, @han-steve, @GeneDer, @denadai2, @thomasdesr, @jaychia, @raulchen, @omatthew98, @srinathk10, @alanwguo, @rueian, @akshay-anyscale, @bveeramani, @dentiny, @dhakshin32, @kouroshHakha, @sven1977

There is no difference between 2.44.1 and 2.44.0, though we needed a patch version for other out of band reasons. To fill the awkward blankness, here is a haiku about Ray:

Under screen-lit skies A ray of bliss in each patch Joy at any scale

Release Highlights

• This release features Ray Compiled Graph (beta). Ray Compiled Graph gives you a classic Ray Core-like API, but with (1) less than 50us system overhead for workloads that repeatedly execute the same task graph; and (2) native support for GPU-GPU communication via NCCL. Ray Compiled Graph APIs simplify high-performance multi-GPU workloads such as LLM inference and training. The beta release refines the API, enhances stability, and adds or improves features like visualization, profiling and experimental GPU compute/computation overlap. For more information, refer to Ray documentation: https://docs.ray.io/en/latest/ray-core/compiled-graph/ray-compiled-graph.html
• The experimental Ray Workflows library has been deprecated and will be removed in a future version of Ray. Ray Workflows has been marked experimental since its inception and hasn’t been maintained due to the Ray team focusing on other priorities. If you are using Ray Workflows, we recommend pinning your Ray version to 2.44.

Ray Libraries

Ray Data

🎉 New Features:

• Add Iceberg write support through pyiceberg (#50590)
• [LLM] Various feature enhancements to Ray Data LLM, including LoRA support #50804 and structured outputs #50901

💫 Enhancements:

• Add dataset/operator state, progress, total metrics (#50770)
• Make chunk combination threshold configurable (#51200)
• Store average memory use per task in OpRuntimeMetrics (#51126)
• Avoid unnecessary conversion to Numpy when creating Arrow/Pandas blocks (#51238)
• Append-mode API for preprocessors -- #50848, #50847, #50642, #50856, #50584. Note that vectorizers and hashers now output a single column instead 1 column per feature. In the near future, we will be graduating preprocessors to beta.

🔨 Fixes:

• Fixing Map Operators to avoid unconditionally overriding generator's back-pressure configuration (#50900)
• Fix filter expr equating negative numbers (#50932)
• Fix error message for override_num_blocks when reading from a HuggingFace Dataset (#50998)
• Make num_blocks in repartition optional (#50997)
• Always pin the seed when doing file-based random shuffle (#50924)
• Fix StandardScaler to handle NaN stats (#51281)

Ray Train

🎉 New Features:

• Implement state export API (#50622, #51085, #51177)

💫 Enhancements:

• Folded v2.XGBoostTrainer API into the public trainer class as an alternate constructor (#50045)
• Created a default ScalingConfig if one is not provided to the trainer (#51093)
• Improved TrainingFailedError message (#51199)
• Utilize FailurePolicy factory (#51067)

🔨 Fixes:

• Fixed trainer import deserialization when captured within a Ray task (#50862)
• Fixed serialize import test for Python 3.12 (#50963)
• Fixed RunConfig deprecation message in Tune being emitted in trainer.fit usage (#51198)

📖 Documentation:

• [Train V2] Updated API references (#51222)
• [Train V2] Updated persistent storage guide (#51202)
• [Train V2] Updated user guides for metrics, checkpoints, results, and experiment tracking (#51204)
• [Train V2] Added updated Train + Tune user guide (#51048)
• [Train V2] Added updated fault tolerance user guide (#51083)
• Improved HF Transformers example (#50896)
• Improved Train DeepSpeed example (#50906)
• Use correct mean and standard deviation norm values in image tutorials (#50240)

🏗 Architecture refactoring:

• Deprecated Torch AMP wrapper utilities (#51066)
• Hid private functions of train context to avoid abuse (#50874)
• Removed ray storage dependency and deprecated RAY_STORAGE env var configuration option (#50872)
• Moved library usage tests out of core (#51161)

Ray Tune

📖 Documentation:

• Various improvements to Tune Pytorch CIFAR tutorial (#50316)
• Various improvements to the Ray Tune XGBoost tutorial (#50455)
• Various enhancements to Tune Keras example (#50581)
• Minor improvements to Hyperopt tutorial (#50697)
• Various improvements to LightGBM tutorial (#50704)
• Fixed non-runnable Optuna tutorial (#50404)
• Added documentation for Asynchronous HyperBand Example in Tune (#50708)
• Replaced reuse actors example with a fuller demonstration (#51234)
• Fixed broken PB2/RLlib example (#51219)
• Fixed typo and standardized equations across the two APIs (#51114)
• Improved PBT example (#50870)
• Removed broken links in documentation (#50995, #50996)

🏗 Architecture refactoring:

• Removed ray storage dependency and deprecated RAY_STORAGE env var configuration option (#50872)
• Moved library usage tests out of core (#51161)

Ray Serve

🎉 New Features:

• Faster bulk imperative Serve Application deploys (#49168)
• [LLM] Add gen-config (#51235)

💫 Enhancements:

• Clean up shutdown behavior of serve (#51009)
• Add additional_log_standard_attrs to serve logging config (#51144)
• [LLM] remove asyncache and cachetools from dependencies (#50806)
• [LLM] remove backoff dependency (#50822)
• [LLM] Remove asyncio_timeout from ray[llm] deps on python<3.11 (#50815)
• [LLM] Made JSON validator a singleton and jsonref packages lazy imported (#50821)
• [LLM] Reuse AutoscalingConfig and DeploymentConfig from Serve (#50871)
• [LLM] Use pyarrow FS for cloud remote storage interaction (#50820)
• [LLM] Add usage telemetry for serve.llm (#51221)

🔨 Fixes:

• Exclude redirects from request error count (#51130)
• [LLM] Fix the wrong device_capability issue in vllm on quantized models (#51007)
• [LLM] add gen-config related data file to the package (#51347)

📖 Documentation:

• [LLM] Fix quickstart serve LLM docs (#50910)
• [LLM] update build_openai_app to include yaml example (#51283)
• [LLM] remove old vllm+serve doc (#51311)

RLlib

💫 Enhancements:

• APPO/IMPALA accelerate:
  • LearnerGroup should not pickle remote functions on each update-call; Refactor LearnerGroup and Learner APIs. (#50665)
  • EnvRunner sync enhancements. (#50918)
  • Various other speedups: #51302, #50923, #50919, #50791
• Unify namings for actor managers' outstanding in-flight requests metrics. (#51159)
• Add timers to env step, forward pass, and complete connector pipelines runs. (#51160)

🔨 Fixes:

• Multi-agent env vectorization:
  • Fix MultiAgentEnvRunner env check bug. (#50891)
  • Add single_action_space and single_observation_space to VectorMultiAgentEnv. (#51096)
• Other fixes: #51255, #50920, #51369

📖 Documentation:

• Smaller fixes: #51015, #51219

Ray Core and Ray Clusters

Ray Core

🎉 New Features:

• Enhanced uv support (#51233)

💫 Enhancements:

• Made infeasible task errors much more obvious (#45909)
• Log rotation for workers, runtime env agent, and dashboard agent (#50759, #50877, #50909)
• Support customizing gloo timeout (#50223)
• Support torch profiling in Compiled Graph (#51022)
• Change default tensor deserialization in Compiled Graph (#50778)
• Use current node id if no node is specified on ray drain-node (#51134)

🔨 Fixes:

• Fixed an issue where the raylet continued to have high CPU overhead after a job was terminated (#49999).
• Fixed compiled graph buffer release issues (#50434)
• Improved logic for ray.wait on object store objects (#50680)
• Ray metrics performing validation the same validation as Prometheus for invalid names (#40586)
• Make executor a long-running Python thread (#51016)
• Fix plasma client memory leak (#51051)
• Fix using ray.actor.exit_actor() from within an async background thread (#49451)
• Fix UV hook to support Ray Job submission (#51150)
• Fix resource leakage after ray job is finished (#49999)
• Use the correct way to check whether an actor task is running (#51158)
• Controllably destroy CUDA events in GPUFuture’s (Compiled Graph) (#51090)
• Avoid creating a thread pool with 0 threads (#50837)
• Fix the logic to calculate the number of workers based on the TPU version (#51227)

📖 Documentation:

• Updated error message and anti-pattern when forking new processes in worker processes (#50705)
• Compiled Graph API Documentation (#50754)
• Doc for nsight and torch profile for Compiled Graph (#51037)
• Compiled Graph Troubleshooting Doc (#51030)
• Completion of of Compiled Graph Docs (#51206)
• Updated jemalloc profiling doc (#51031)
• Add information about standard Python logger attributes (#51038)
• Add description for named placement groups to require a namespace (#51285)
• Deprecation warnings for Ray Workflows and cluster-wide storage (#51309)

Ray Clusters

🎉 New Features:

• Add cuda 12.8 images (#51210)

💫 Enhancements:

• Add Pod names to the output of ray status -v (#51192)

🔨 Fixes:

• Fix autoscaler v1 crash from infeasible strict spread placement groups (#39691)

🏗 Architecture refactoring:

• Refactor autoscaler v2 log formatting (#49350)
• Update yaml example for CoordinatorSenderNodeProvider (#51292)

Dashboard

🎉 New Features:

• Discover TPU logs on the Ray Dashboard (#47737)

🔨 Fixes:

• Return the correct error message when trying to kill non-existent actors (#51341)

Many thanks to all those who contributed to this release! @crypdick, @rueian, @justinvyu, @MortalHappiness, @CheyuWu, @GeneDer, @dayshah, @lk-chen, @matthewdeng, @co63oc, @win5923, @sven1977, @akshay-anyscale, @ShaochenYu-YW, @gvspraveen, @bveeramani, @jakac, @VamshikShetty, @raulchen, @PaulFenton, @elimelt, @comaniac, @qinyiyan, @ruisearch42, @nadongjun, @AndyUB, @israbbani, @hongpeng-guo, @laysfire, @alexeykudinkin, @Drice1999, @harborn, @scottsun94, @abrarsheikh, @martinbomio, @MengjinYan, @HollowMan6, @orcahmlee, @kenchung285, @csy1204, @noemotiovon, @jujipotle, @davidxia, @kevin85421, @hcc429, @edoakes, @kouroshHakha, @omatthew98, @alanwguo, @farridav, @aslonnie, @simonsays1980, @pcmoritz, @terraflops1048576, @JoshKarpel, @SumanthRH, @sijieamoy, @zcin, @can-anyscale, @akyang-anyscale, @angelinalg, @saihaj, @jjyao, @anmscale, @ryanaoleary, @dentiny, @jimmyxie-figma, @stephanie-wang, @khluu, @maofagui

Highlights

• This release features new modules in Ray Serve and Ray Data for integration with large language models, marking the first step of addressing #50639. Existing Ray Data and Ray Serve have limited support for LLM deployments, where users have to manually configure and manage the underlying LLM engine. In this release, we offer APIs for both batch inference and serving of LLMs within Ray in ray.data.llm and ray.serve.llm. See the below notes for more details. These APIs are marked as alpha -- meaning they may change in future releases without a deprecation period.
• Ray Train V2 is available to try starting in Ray 2.43! Run your next Ray Train job with the RAY_TRAIN_V2_ENABLED=1 environment variable. See the migration guide for more information.
• A new integration with uv run that allows easily specifying Python dependencies for both driver and workers in a consistent way and enables quick iterations for development of Ray applications (#50160, 50462), check out our blog post

Ray Libraries

Ray Data

🎉 New Features:

• Ray Data LLM: We are introducing a new module in Ray Data for batch inference with LLMs (currently marked as alpha). It offers a new Processor abstraction that interoperates with existing Ray Data pipelines. This abstraction can be configured two ways:
  • Using the vLLMEngineProcessorConfig, which configures vLLM to load model replicas for high throughput model inference
  • Using the HttpRequestProcessorConfig, which sends HTTP requests to an OpenAI-compatible endpoint for inference.
  • Documentation for these features can be found here.
• Implement accurate memory accounting for UnionOperator (#50436)
• Implement accurate memory accounting for all-to-all operations (#50290)

💫 Enhancements:

• Support class constructor args for filter() (#50245)
• Persist ParquetDatasource metadata. (#50332)
• Rebasing ShufflingBatcher onto try_combine_chunked_columns (#50296)
• Improve warning message if required dependency isn't installed (#50464)
• Move data-related test logic out of core tests directory (#50482)
• Pass executor as an argument to ExecutionCallback (#50165)
• Add operator id info to task+actor (#50323)
• Abstracting common methods, removing duplication in ArrowBlockAccessor, PandasBlockAccessor (#50498)
• Warn if map UDF is too large (#50611)
• Replace AggregateFn with AggregateFnV2, cleaning up Aggregation infrastructure (#50585)
• Simplify Operator.repr (#50620)
• Adding in TaskDurationStats and on_execution_step callback (#50766)
• Print Resource Manager stats in release tests (#50801)

🔨 Fixes:

• Fix invalid escape sequences in grouped_data.py docstrings (#50392)
• Deflake test_map_batches_async_generator (#50459)
• Avoid memory leak with pyarrow.infer_type on datetime arrays (#50403)
• Fix parquet partition cols to support tensors types (#50591)
• Fixing aggregation protocol to be appropriately associative (#50757)

📖 Documentation:

• Remove "Stable Diffusion Batch Prediction with Ray Data" example (#50460)

Ray Train

🎉 New Features:

• Ray Train V2 is available to try starting in Ray 2.43! Run your next Ray Train job with the RAY_TRAIN_V2_ENABLED=1 environment variable. See the migration guide for more information.

💫 Enhancements:

• Add a training ingest benchmark release test (#50019, #50299) with a fault tolerance variant (#50399)
• Add telemetry for Trainer usage in V2 (#50321)
• Add pydantic as a ray[train] extra install (#46682)
• Add state tracking to train v2 to make run status, run attempts, and training worker metadata observable (#50515)

🔨 Fixes:

• Increase doc test parallelism (#50326)
• Disable TF test for py312 (#50382)
• Increase test timeout to deflake (#50796)

📖 Documentation:

• Add missing xgboost pip install in example (#50232)

🏗 Architecture refactoring:

• Add deprecation warnings pointing to a migration guide for Ray Train V2 (#49455, #50101, #50322)
• Refactor internal Train controller state management (#50113, #50181, #50388)

Ray Tune

🔨 Fixes:

• Fix worker node failure test (#50109)

📖 Documentation:

• Update all doc examples off of ray.train imports (#50458)
• Update all ray/tune/examples off of ray.train imports (#50435)
• Fix typos in persistent storage guide (#50127)
• Remove Binder notebook links in Ray Tune docs (#50621)

🏗 Architecture refactoring:

• Update RLlib to use ray.tune imports instead of ray.air and ray.train (#49895)

Ray Serve

🎉 New Features:

• Ray Serve LLM: We are introducing a new module in Ray Serve to easily integrate open source LLMs in your Ray Serve deployment, currently marked as alpha. This opens up a powerful capability of composing complex applications with multiple LLMs, which is a use case in emerging applications like agentic workflows. Ray Serve LLM offers a couple core components, including:
  • VLLMService: A prebuilt deployment that offers a full-featured vLLM engine integration, with support for features such as LoRA multiplexing and multimodal language models.
  • LLMRouter: An out-of-the-box OpenAI compatible model router that can route across multiple LLM deployments.
  • Documentation can be found at https://docs.ray.io/en/releases-2.43.0/serve/llm/overview.html

💫 Enhancements:

• Add required_resources to REST API (#50058)

🔨 Fixes:

• Fix batched requests hanging after cancellation (#50054)
• Properly propagate backpressure error (#50311)

RLlib

🎉 New Features:

• Added env vectorization support for multi-agent (new API stack). (#50437)

💫 Enhancements:

• APPO/IMPALA various acceleration efforts. Reached 100k ts/sec on Atari benchmark with 400 EnvRunners and 16 (multi-node) GPU Learners: #50760, #50162, #50249, #50353, #50368, #50379, #50440, #50477, #50527, #50528, #50600, #50309
• Offline RL:
  • Remove all weight synching to eval_env_runner_group from the training steps. (#50057)
  • Enable single-learner/multi-learner GPU training. (#50034)
  • Remove reference to MARWILOfflinePreLearner in OfflinePreLearner docstring. (#50107)
  • Add metrics to multi-agent replay buffers. (#49959)

🔨 Fixes:

• Fix SPOT preemption tolerance for large AlgorithmConfig: Pass by reference to RolloutWorker (#50688)
• on_workers/env_runners_recreated callback would be called twice. (#50172)
• default_resource_request: aggregator actors missing in placement group for local Learner. (#50219, #50475)

📖 Documentation:

• Docs re-do (new API stack):
  • Rewrite/enhance "getting started" rst page. (#49950)
  • Remove rllib-models.rst and fix broken html links. (#49966, #50126)