Advanced

AI Risk Documentation

Comprehensive documentation is the backbone of AI risk management. It provides accountability, enables auditing, supports regulatory compliance, and ensures institutional knowledge persists across teams.

AI Risk Register

A risk register is the central repository for all identified AI risks, their assessments, and mitigation status:

Register Field Purpose Update Frequency
Risk ID & Description Unique identifier and clear description of each risk At identification
Risk Category Classification per taxonomy (technical, ethical, legal, etc.) At identification
Likelihood & Impact Score Quantified risk level using organization's scoring methodology Quarterly or at trigger events
Mitigation Controls Description of implemented and planned controls As controls change
Risk Owner Person accountable for managing and monitoring the risk At assignment
Residual Risk Risk level remaining after controls are applied After control implementation
Status Open, mitigated, accepted, transferred, or closed At each review

Algorithmic Impact Assessments

  1. System Description

    Document the AI system's purpose, capabilities, limitations, intended users, affected populations, and deployment context. Include technical architecture and data flow diagrams.

  2. Rights Impact Analysis

    Assess potential impacts on fundamental rights including non-discrimination, privacy, freedom of expression, and access to services. Map affected populations and vulnerable groups.

  3. Proportionality Assessment

    Evaluate whether the AI system's benefits are proportionate to its risks. Consider whether less risky alternatives could achieve similar outcomes.

  4. Mitigation and Monitoring Plan

    Document specific measures to address identified impacts, including monitoring indicators, review schedules, and escalation procedures.

Model Cards and Datasheets

⚠
Regulatory Trend: The EU AI Act requires extensive technical documentation for high-risk AI systems. Model cards and datasheets, while originating in research, are increasingly becoming regulatory expectations. Start creating them now to build institutional capability.

Model Cards

Document model details (architecture, training procedure), intended use, performance metrics across demographic groups, limitations, ethical considerations, and maintenance information. Follow the Mitchell et al. framework.

Datasheets for Datasets

Document dataset motivation, composition, collection process, preprocessing, distribution, maintenance, and legal/ethical considerations. Follow the Gebru et al. framework for comprehensive data documentation.

System Cards

Document the entire AI system including model, data pipeline, human oversight processes, deployment architecture, and monitoring setup. System cards capture the full context that model cards alone cannot.

Decision Logs

Record key decisions made during AI development: why specific training data was chosen, what fairness metrics were prioritized and why, what risks were accepted and by whom.

💡
Next Up: In the final lesson, we cover best practices for building a sustainable risk management culture, integrating risk into MLOps, and scaling processes across the organization.