Lilly Tech Systems
Intermediate
Savings Plans for AI Workloads
Understand and choose between Compute Savings Plans, EC2 Instance Savings Plans, and SageMaker Savings Plans for your ML infrastructure.
Savings Plan Types Comparison
| Feature | Compute SP | EC2 Instance SP | SageMaker SP |
|---|---|---|---|
| Savings | Up to 66% | Up to 72% | Up to 64% |
| Flexibility | Any region, family, OS | Specific family & region | Any region, family |
| Applies to | EC2, Fargate, Lambda | EC2 only | SageMaker only |
| Terms | 1 or 3 year | 1 or 3 year | 1 or 3 year |
| Best for | Mixed ML workloads | Stable GPU fleets | SageMaker endpoints |
Choosing the Right Plan
Compute Savings Plans
Best for teams that may change instance types as new GPUs launch. Covers EC2, EKS, and Fargate workloads across regions.
EC2 Instance Plans
Best when you're committed to a specific GPU family (e.g., p4d) in a specific region for at least a year.
SageMaker Plans
Best for production inference endpoints on SageMaker that run 24/7. Covers notebooks, training, and endpoints.
Calculating Your Commitment
- Analyze baseline usage: Use Cost Explorer to identify your minimum sustained compute usage over the past 3 months
- Cover the floor: Commit only to your minimum baseline; use On-Demand and Spot for variable workloads
- Start conservative: Begin with a smaller commitment and add more as your usage patterns stabilize
- Layer commitments: Stack multiple smaller Savings Plans that expire at different times for flexibility
- Review quarterly: Reassess utilization and adjust future commitments based on actual usage
Good to know: AWS Cost Explorer provides Savings Plan recommendations based on your historical usage. Access it via the AWS Console under Cost Management. The recommendations show projected savings for different commitment levels and terms.
Pro tip: For AI workloads, Compute Savings Plans offer the best balance of savings and flexibility. Since GPU instance families evolve rapidly (from P4 to P5, or the shift to Inferentia), you want the flexibility to switch without losing your commitment. Only lock into EC2 Instance Savings Plans if you're confident in your instance family choice for the full term.