DeepSpeed

Master Microsoft DeepSpeed: train trillion-parameter models. Learn ZeRO stages, optimizer offload, pipeline parallelism, and the patterns for memory-efficient training.

Start Topic → View All Lessons

6

Lessons

💻

Code Examples

✅

Production-Ready

100%

Free

Lessons in This Topic

Work through these 6 lessons in order, or jump to whichever topic you need most.

DeepSpeed Overview

Intermediate

ZeRO Stages 1, 2, 3

Advanced

ZeRO-Offload to CPU/NVMe

Advanced

Pipeline Parallelism

Advanced

DeepSpeed Config Files

Intermediate

DeepSpeed vs FSDP

Advanced