Lilly Tech Systems
Beginner
Introduction to Azure Kubernetes for AI
Understand why Azure Kubernetes Service (AKS) is a powerful platform for running AI workloads and how it compares to Azure ML managed compute.
Why AKS for AI?
Full Control
Complete control over your serving stack, scheduling, networking, and infrastructure configuration.
Ecosystem
Access to the entire Kubernetes ecosystem: KServe, Triton, Ray, Argo Workflows, and custom operators.
Multi-Workload
Run training, inference, data processing, and application services on the same cluster infrastructure.
Portability
Kubernetes workloads are portable across clouds, avoiding vendor lock-in on the serving layer.
AKS vs Azure ML Compute
| Feature | AKS | Azure ML Compute |
|---|---|---|
| Management | You manage K8s | Fully managed |
| Flexibility | Full customization | Azure ML patterns |
| Serving frameworks | Any (Triton, KServe, etc.) | Azure ML Endpoints |
| Scaling | HPA, KEDA, custom | Auto-scale rules |
| Best for | K8s-native teams, custom infra | Azure-native ML teams |
Good to know: AKS and Azure ML are complementary. You can attach an AKS cluster to an Azure ML workspace to use Azure ML for experiment tracking and model registry while serving models on your own AKS infrastructure with custom serving frameworks.
Key takeaway: Choose AKS when your team has Kubernetes expertise and needs full control over the ML serving stack, custom scheduling, or multi-framework support. Choose Azure ML managed compute when you want a fully managed experience with less infrastructure overhead.