Responsible AI (5-10%) Advanced

Although this is the lowest-weighted domain, it can make the difference between passing and failing. Microsoft strongly emphasizes responsible AI practices. This lesson covers fairness assessment, model interpretability, differential privacy, error analysis, and the Responsible AI dashboard — all testable on DP-100.

Microsoft's Responsible AI Principles

Microsoft defines six core principles. Know these for the exam — they often appear in scenario-based questions.

The Responsible AI Dashboard

The Responsible AI dashboard is an integrated tool in Azure ML Studio that combines multiple analysis components into a single view. This is heavily tested on the exam.

Dashboard Components

# Create a Responsible AI dashboard
from azure.ai.ml.entities import (
    ResponsibleAiInsights,
    RAIInsightsConfig
)

# Define RAI components to include
rai_config = RAIInsightsConfig(
    components=[
        # Error analysis
        {"type": "error_analysis", "max_depth": 3, "num_leaves": 31},
        # Model explanations (feature importance)
        {"type": "explanation"},
        # Counterfactual analysis
        {"type": "counterfactual",
         "total_counterfactuals": 10,
         "desired_class": "opposite"},
        # Causal analysis
        {"type": "causal",
         "treatment_features": ["monthly_charges", "contract_type"]}
    ]
)

# Submit as a pipeline job
from azure.ai.ml import dsl

@dsl.pipeline(compute="dp100-cluster")
def rai_pipeline():
    # Gather RAI insights
    rai_job = rai_insights_component(
        model=Input(type="mlflow_model", path="azureml:churn-model:1"),
        train_dataset=Input(type="mltable", path="azureml:churn-train:1"),
        test_dataset=Input(type="mltable", path="azureml:churn-test:1"),
        target_column_name="churn",
        task_type="classification"
    )
    return rai_job.outputs

pipeline_job = ml_client.jobs.create_or_update(rai_pipeline())

Fairness Assessment

Fairness analysis checks whether your model performs equally across different demographic groups (sensitive features like gender, age, race).

Key Fairness Metrics

# Fairness assessment with Fairlearn (integrated in Azure ML)
from fairlearn.metrics import MetricFrame
from sklearn.metrics import accuracy_score, recall_score

# Assume y_true, y_pred, and sensitive_feature are defined
metric_frame = MetricFrame(
    metrics={
        "accuracy": accuracy_score,
        "recall": recall_score
    },
    y_true=y_test,
    y_pred=predictions,
    sensitive_features=test_df["gender"]   # Sensitive attribute
)

# Overall metrics
print("Overall metrics:")
print(metric_frame.overall)

# Metrics by group
print("\nMetrics by group:")
print(metric_frame.by_group)

# Difference between best and worst group
print("\nMax disparity:")
print(metric_frame.difference(method="between_groups"))

Model Interpretability

Interpretability explains why a model made a specific prediction. Azure ML integrates with InterpretML for both global and local explanations.

• Global explanations — Which features are most important overall? (e.g., "monthly_charges is the top predictor of churn")
• Local explanations — Why was this specific prediction made? (e.g., "Customer X was predicted to churn because their monthly charges are $95 and contract is month-to-month")

# Model interpretability with InterpretML
from interpret.ext.blackbox import TabularExplainer

# Create explainer
explainer = TabularExplainer(
    model,
    X_train,
    features=feature_names,
    classes=["No Churn", "Churn"]
)

# Global explanation (overall feature importance)
global_explanation = explainer.explain_global(X_test)
print("Top features:", global_explanation.get_feature_importance_dict())

# Local explanation (single prediction)
local_explanation = explainer.explain_local(X_test.iloc[[0]])
print("Local importance:", local_explanation.get_ranked_local_names())
print("Local values:", local_explanation.get_ranked_local_values())

Differential Privacy

Differential privacy adds mathematical guarantees that individual records cannot be identified from model outputs. Azure ML supports SmartNoise for differential privacy.

Error Analysis

Error analysis identifies where your model fails most. Instead of looking at overall accuracy, it finds cohorts (subsets) with high error rates.

# Error analysis concepts for the exam
# The Error Analysis component creates:

# 1. Error Tree: A decision tree that splits data by features
#    to find cohorts with highest error rates
#    Example: "Customers with tenure < 6 months AND monthly_charges > $80
#    have a 45% error rate (vs 12% overall)"

# 2. Error Heatmap: A 2D grid showing error rates
#    across pairs of features
#    Example: Rows = tenure buckets, Columns = contract type
#    Cell color = error rate for that combination

# Key insight: Error analysis helps you decide:
# - Which cohorts need more training data
# - Whether to build specialized models for high-error groups
# - Where to set different confidence thresholds

Practice Questions