Intermediate

Bias Detection

Before you can fix bias, you need to measure it. Learn the key fairness metrics and statistical methods for detecting bias in your AI systems.

Key Fairness Metrics

Metric Definition When to Use
Demographic Parity The probability of a positive outcome should be equal across groups: P(Y=1|A=0) = P(Y=1|A=1) When equal selection rates are required regardless of qualification differences
Equalized Odds True positive rate and false positive rate should be equal across groups When both types of errors should be equally distributed across groups
Equal Opportunity True positive rate should be equal across groups (relaxed version of equalized odds) When equal access to positive outcomes among qualified individuals is the priority
Disparate Impact Ratio of positive outcome rates between groups. The four-fifths rule requires this ratio to be at least 0.8 Legal compliance in employment and lending decisions
Calibration Among individuals assigned the same score, the actual positive rate should be equal across groups When the model produces probability scores that are used for decision-making
⚠
Impossibility Theorem: It is mathematically impossible to satisfy all fairness metrics simultaneously (except in trivial cases). You must choose which fairness criteria are most important for your specific context and use case.

Disparate Impact Analysis

Python - Calculating Disparate Impact 
import numpy as np

def disparate_impact(y_pred, protected_attribute):
    """Calculate disparate impact ratio (four-fifths rule)."""
    # Positive rate for unprivileged group
    unpriv_mask = protected_attribute == 0
    unpriv_rate = y_pred[unpriv_mask].mean()

    # Positive rate for privileged group
    priv_mask = protected_attribute == 1
    priv_rate = y_pred[priv_mask].mean()

    # Disparate impact ratio
    di_ratio = unpriv_rate / priv_rate if priv_rate > 0 else 0

    # Four-fifths rule: DI ratio should be >= 0.8
    passes_rule = di_ratio >= 0.8

    return {
        "di_ratio": round(di_ratio, 3),
        "passes_four_fifths": passes_rule,
        "privileged_rate": round(priv_rate, 3),
        "unprivileged_rate": round(unpriv_rate, 3)
    }

Equalized Odds Analysis

Python - Equalized Odds Check 
def equalized_odds(y_true, y_pred, protected_attribute):
    """Check equalized odds across groups."""
    results = {}
    for group in [0, 1]:
        mask = protected_attribute == group
        y_t = y_true[mask]
        y_p = y_pred[mask]

        # True Positive Rate
        pos_mask = y_t == 1
        tpr = y_p[pos_mask].mean() if pos_mask.sum() > 0 else 0

        # False Positive Rate
        neg_mask = y_t == 0
        fpr = y_p[neg_mask].mean() if neg_mask.sum() > 0 else 0

        results[f"group_{group}"] = {"TPR": round(tpr, 3), "FPR": round(fpr, 3)}

    # Check if differences exceed threshold
    tpr_diff = abs(results["group_0"]["TPR"] - results["group_1"]["TPR"])
    fpr_diff = abs(results["group_0"]["FPR"] - results["group_1"]["FPR"])

    results["equalized"] = tpr_diff < 0.05 and fpr_diff < 0.05
    return results

Detection Approaches by Pipeline Stage

Data Auditing

Analyze training data demographics, label distributions, and feature correlations with protected attributes before training begins.

Model Evaluation

Compute fairness metrics on held-out test sets, disaggregated by protected groups. Use slice-based evaluation to find hidden disparities.

Production Monitoring

Track fairness metrics in real time on production data. Set up alerts when metrics drift beyond acceptable thresholds.

Counterfactual Testing

Change protected attributes in test inputs and check if the model's output changes. If it does, the model is using protected information.