Ray 2.20.0

Ray Libraries

Ray Data

💫 Enhancements:

• Dedupe repeated schema during ParquetDatasource metadata prefetching (#44750)
• Update map_groups implementation to better handle large outputs (#44862)
• Deprecate prefetch_batches arg of iter_rows and change default value (#44982)
• Adding in default behavior to false for creating dirs on s3 writes (#44972)
• Make internal UDF names more descriptive (#44985)
• Make name a required argument for AggregateFn (#44880)

📖 Documentation:

• Add key concepts to and revise "Data Internals" page (#44751)

Ray Train

💫 Enhancements:

• Setup XGBoost CommunicatorContext automatically (#44883)
• Track Train Run Info with TrainStateActor (#44585)

📖 Documentation:

• Add documentation for accelerator_type (#44882)
• Update Ray Train example titles (#44369)

Ray Tune

💫 Enhancements:

• Remove trial table when running Ray Train in a Jupyter notebook (#44858)
• Clean up temporary checkpoint directories for class Trainables (ex: RLlib) (#44366)

📖 Documentation:

• Fix minor doc format issues (#44865)
• Remove outdated ScalingConfig references (#44918)

Ray Serve

💫 Enhancements:

• Handle push metric interval is now configurable with environment variable RAY_SERVE_HANDLE_METRIC_PUSH_INTERVAL_S (#32920)
• Improve performance of developer API serve.get_app_handle (#44812)

🔨 Fixes:

• Fix memory leak in handles for autoscaling deployments (the leak happens when
• RAY_SERVE_COLLECT_AUTOSCALING_METRICS_ON_HANDLE=1) (#44877)

RLlib

🎉 New Features:

• Introduce MetricsLogger, a unified API for users of RLlib to log custom metrics and stats in all of RLlib’s components (Algorithm, EnvRunners, and Learners). Rolled out for new API stack for Algorithm (training_step) and EnvRunners (custom callbacks). Learner (custom loss functions) support in progress. #44888, #44442
• Introduce “inference-only” (slim) mode for RLModules that run inside an EnvRunner (and thus don’t require value-functions or target networks): #44797

💫 Enhancements:

• MultiAgentEpisodeReplayBuffer for new API stack (preparation for multi-agent support of SAC and DQN): #44450
• AlgorithmConfig cleanup and renaming of properties and methods for better consistency/transparency: #44896

🔨 Fixes:

• Various minor bug fixes: #44989, #44988, #44891, #44898, #44868, #44867, #44845

Ray Core and Ray Clusters

💫 Enhancements:

• Report GCS internal pubsub buffer metrics and cap message size (#44749)

🔨 Fixes:

• Fix task submission never return when network partition happens (#44692)
• Fix incorrect use of ssh port forward option. (#44973)
• Make sure dashboard will exit if grpc server fails (#44928)
• Make sure dashboard agent will exit if grpc server fails (#44899)

Thanks @can-anyscale, @hongchaodeng, @zcin, @marwan116, @khluu, @bewestphal, @scottjlee, @andrewsykim, @anyscalesam, @MortalHappiness, @justinvyu, @JoshKarpel, @woshiyyya, @rynewang, @Abirdcfly, @omatthew98, @sven1977, @marcelocarmona, @rueian, @mattip, @angelinalg, @aslonnie, @matthewdeng, @abizjakpro, @simonsays1980, @jjyao, @terraflops1048576, @hongpeng-guo, @stephanie-wang, @bw-matthew, @bveeramani, @ruisearch42, @kevin85421, @Tongruizhe

Many thanks to all those who contributed to this release!