# Ray 2.4.0 — Ray-2.4.0

- Product: Ray (https://whatsnew.fyi/product/ray)
- Vendor: Anyscale
- Date: 2023-04-25
- Version: 2.4.0
- Original notes: https://github.com/ray-project/ray/releases/tag/ray-2.4.0
- Permalink: https://whatsnew.fyi/product/ray/releases/2.4.0

What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog.

Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'.

---

- **added** — Introduce LightningTrainer to scale PyTorch Lightning on Ray
- **added** — Release AccelerateTrainer to run HuggingFace Accelerate and DeepSpeed on Ray
- **added** — Add examples for GPT-J fine-tuning with Microsoft DeepSpeed and Ray Train
- **added** — Add examples for GPT-J-6B batch prediction with Ray Data
- **added** — Add examples for GPT-J-6B serving with Ray Serve
- **added** — Add examples for Stable Diffusion Dreambooth fine-tuning with Ray Train
- **added** — Add examples for Stable Diffusion batch prediction with Ray Data
- **added** — Add examples for Stable Diffusion serving with Ray Serve
- **added** — Enable streaming execution by default for more efficient data processing pipelines
- **added** — Implement asynchronous batch prefetching in Dataset.iter_batches
- **added** — Add support for reading SQL databases with Ray Data
- **added** — Add support for reading WebDataset format
- **added** — Add multi-app CLI and REST API support to Ray Serve
- **added** — Add TorchDetectionPredictor
- **added** — Add collate_fn parameter to iter_torch_batches
- **changed** — Make Preprocessor.transform lazy by default
- **changed** — Make BatchPredictor lazy
- **changed** — Enhance logging and metrics for Serve applications
- **changed** — Replace ray_lightning integration with LightningTrainer for latest PyTorch Lightning compatibility
- **deprecated** — Ray 2.4 is the last version that supports Python 3.6

#### Ray 2.4 - Generative AI and LLM support
Over the last few months, we have seen a flurry of innovative activity around [generative AI models](https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-generative-ai) and [large language models (LLM)](https://en.wikipedia.org/wiki/Large_language_model). To continue our effort to ensure Ray provides a pivotal compute substrate for [generative AI workloads](https://www.anyscale.com/blog/ray-common-production-challenges-for-generative-ai-infrastructure) and addresses the challenges (as explained in our [blog series](https://www.anyscale.com/blog/ray-common-production-challenges-for-generative-ai-infrastructure)), we have invested engineering efforts in this release to ensure that these open source LLM models and workloads are accessible to the open source community and performant with Ray. 

This release includes new examples for training, batch inference, and serving with your own LLM.

##### Generative AI and LLM Examples
* [GPT-J (LLM) fine-tuning with Microsoft DeepSpeed and Ray Train](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/gptj_deepspeed_fine_tuning.html)  
* [GPT-J-6B Batch Prediction with Ray Data](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/gptj_batch_prediction.html)
* [GPT-J-6B Serving with Ray Serve](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/gptj_serving.html) 
* [Stable Diffusion (Dreambooth) fine-tuning with Ray Train](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/dreambooth_finetuning.html)
* [Stable Diffusion Batch Prediction with Ray Data ](https://docs.ray.io/en/releases-2.4.0/ray-air/examples/stablediffusion_batch_prediction.html)
* [Stable Diffusion Serving with Ray Serve](https://docs.ray.io/en/releases-2.4.0/serve/tutorials/stable-diffusion.html)

##### Ray Train enhancements
  * We're introducing the [LightningTrainer](https://docs.ray.io/en/releases-2.4.0/train/api/doc/ray.train.lightning.LightningTrainer.html), allowing you to scale your [PyTorch Lightning](https://lightning.ai/docs/pytorch/stable//index.html) on Ray. As part of our continued effort for seamless integration and ease of use, we have enhanced and replaced our existing ray_lightning integration, which was widely adopted, with the latest changes to Pytorch Lighting. 
  * we’re releasing an [AccelerateTrainer](https://docs.ray.io/en/releases-2.4.0/train/api/doc/ray.train.huggingface.accelerate.AccelerateTrainer.html), allowing you to run [HuggingFace Accelerate](https://huggingface.co/docs/accelerate) and [DeepSpeed](https://huggingface.co/docs/accelerate/usage_guides/deepspeed) on Ray with minimal code changes. This Trainer integrates with the rest of the Ray ecosystem—including the ability to run distributed [hyperparameter tuning](https://docs.ray.io/en/latest/tune/index.html) with each trial being a distributed training job.

##### Ray Data highlights
  * Streaming execution is enabled by default, providing users with a more efficient data processing pipeline that can handle larger datasets and minimize memory consumption. Check out the docs here: ([doc](https://docs.ray.io/en/releases-2.4.0/data/dataset-internals.html#streaming-execution)) 
    * Note that this means data output may no longer preserve the original order in more cases. To retain the original ordering properties of Ray Data prior to 2.4, you can set the config `ray.data.DatasetContext.get_current().execution_options.preserve_order = True`.
  * We've implemented asynchronous batch prefetching of Dataset.iter_batches ([doc](https://docs.ray.io/en/releases-2.4.0/data/api/doc/ray.data.DatasetIterator.iter_batches.html)), improving performance by fetching data in parallel while the main thread continues processing, thus reducing waiting time.
  * Support reading SQL databases ([doc](https://docs.ray.io/en/releases-2.4.0/data/creating-datasets.html#reading-from-sql-databases)), enabling users to seamlessly integrate relational databases into their

_[Truncated at 4000 characters — full notes: https://github.com/ray-project/ray/releases/tag/ray-2.4.0]_