Lilly Tech Systems
AI Safety & Alignment
Understand the critical challenges of building AI systems that behave as intended. Learn about alignment theory, RLHF, red teaming, guardrails, and industry best practices for responsible AI development.
Your Learning Path
Follow these lessons in order, or jump to any topic that interests you.
1. Introduction
What is AI safety? Why alignment matters, historical incidents, and the landscape of AI risk research.
2. The Alignment Problem
Specification gaming, reward hacking, mesa-optimization, inner alignment, and the difficulty of defining objectives.
3. RLHF
Reinforcement Learning from Human Feedback — how it works, its strengths, limitations, and alternatives like DPO and RLAIF.
4. Red Teaming
Adversarial testing of AI systems, structured red team methodologies, automated red teaming, and evaluation frameworks.
5. Guardrails
Input/output filtering, content moderation, safety classifiers, constitutional AI, and runtime safety layers.
6. Best Practices
Building a safety culture, evaluation frameworks, incident response, responsible deployment, and staying current.
What You'll Learn
By the end of this course, you'll be able to:
Understand Alignment
Explain the core challenges of aligning AI systems with human intentions, values, and safety requirements.
Apply RLHF Concepts
Understand how RLHF trains models to follow instructions and why it is a key safety technique.
Conduct Red Teaming
Plan and execute adversarial testing sessions to find failure modes before they reach users.
Implement Guardrails
Design and deploy safety layers that protect users and prevent harmful AI outputs in production.