Lilly Tech Systems
Ray on Kubernetes
Deploy and manage Ray clusters on Kubernetes with KubeRay. Learn distributed training with Ray Train, scalable model serving with Ray Serve, and production best practices for running AI workloads at scale.
Your Learning Path
Follow these lessons in order, or jump to any topic that interests you.
1. Introduction
What is Ray, its distributed computing model, and why it's the go-to framework for scaling AI workloads.
2. KubeRay
Install KubeRay operator, understand CRDs (RayCluster, RayJob, RayService), and deploy your first Ray cluster.
3. Ray Clusters
Configure head and worker nodes, set up autoscaling, manage GPU resources, and handle heterogeneous clusters.
4. Ray Train
Distributed training with Ray Train for PyTorch, TensorFlow, and Hugging Face models across GPU clusters.
5. Ray Serve
Deploy models with Ray Serve for scalable online inference, request batching, and model composition.
6. Best Practices
Production patterns for monitoring, fault tolerance, resource optimization, and multi-tenant Ray clusters.
What You'll Learn
By the end of this course, you'll be able to:
Deploy Ray on K8s
Install and configure KubeRay with autoscaling Ray clusters on any Kubernetes environment.
Distributed Training
Scale model training across multiple GPUs and nodes using Ray Train with PyTorch and Hugging Face.
Model Serving
Deploy scalable inference endpoints with Ray Serve featuring batching, composition, and autoscaling.
Production Operations
Monitor, troubleshoot, and optimize Ray clusters for reliability and cost efficiency in production.