Predictive Parity

A practical guide to predictive parity for AI engineers and policymakers.

What This Lesson Covers

Predictive Parity is a key topic in Algorithmic Fairness Metrics. In this lesson you will learn the underlying concept, why it matters specifically for AI engineers and policymakers, the practical approach experienced teams use, and the patterns to avoid. By the end you will be able to engage with predictive parity in real product and policy decisions.

This lesson belongs to the Bias & Fairness category of the AI Ethics & Governance track. Ethics and governance are not optional add-ons — they shape what AI products are allowed to exist, what markets they can enter, and whether the underlying business model holds up under scrutiny.

Why It Matters

Master the math of algorithmic fairness. Learn demographic parity, equalized odds, predictive parity, calibration, and the impossibility theorems.

The reason predictive parity deserves dedicated attention is that the gap between AI teams that take ethics and governance seriously and those that don't is widening fast. Two teams shipping similar products can end up in very different positions when regulators, journalists, customers, or affected communities ask the hard questions. Ethics and governance done well are competitive advantages — not just compliance burdens.

How It Works in Practice

Below is a practical example of how to apply predictive parity in real AI work. Read it once, then think about how you would adapt it to your specific product, regulatory environment, and stakeholders.

# Algorithmic fairness metrics with Fairlearn
from fairlearn.metrics import (
    MetricFrame, demographic_parity_difference,
    equalized_odds_difference, selection_rate,
)
from sklearn.metrics import accuracy_score, false_positive_rate, true_positive_rate
import pandas as pd

# y_pred = model predictions; sensitive_features = protected attribute
metrics = MetricFrame(
    metrics={
        "accuracy":   accuracy_score,
        "selection":  selection_rate,
        "fpr":        false_positive_rate,
        "tpr":        true_positive_rate,
    },
    y_true=y_test,
    y_pred=y_pred,
    sensitive_features=sensitive_features,
)

print(metrics.by_group)             # per-group metrics
print(metrics.difference())          # max - min difference

# Standard fairness criteria
print("Demographic parity diff:", demographic_parity_difference(y_test, y_pred, sensitive_features=sensitive_features))
print("Equalized odds diff:    ", equalized_odds_difference(y_test, y_pred, sensitive_features=sensitive_features))

# Rule of thumb (US): selection rate ratio >= 0.8 (four-fifths rule)

Step-by-Step Walkthrough

1. Identify the affected stakeholders — Not just users. Affected non-users, regulators, employees, and society at large all have stakes in AI decisions. Ethics is about who is in the room, not just whose voice is loudest.
2. Ground the decision in a framework — Pick one: NIST AI RMF, ISO 42001, EU AI Act risk categorization, or your internal ethics framework. Ungrounded debate goes in circles.
3. Get the inputs — Data on bias, customer feedback, regulator signals, comparable cases. Decisions made without inputs are guesses.
4. Document the decision and the reasoning — Future-you and future regulators will want to know what you decided and why. Architecture Decision Records (ADRs) work well.
5. Build in re-review cadence — Ethics norms shift faster than code. Set a calendar reminder to re-evaluate at 6 months, 12 months, and after every material change.

When To Use It (and When Not To)

Predictive Parity applies when:

• The AI feature touches people in consequential ways (jobs, money, freedom, health)
• You operate in a regulated market or one likely to be regulated soon
• The use case involves protected characteristics, vulnerable populations, or public interest
• The cost of getting it wrong (in trust, lawsuits, or harm) outweighs the cost of doing it right

It is the wrong move when:

• A simpler approach (a different feature, a different framing) avoids the ethics challenge entirely
• You are still iterating on whether the feature should exist at all — decide that first
• You are using ethics as a smokescreen to delay shipping a feature you privately know is fine
• The decision is being made unilaterally by people without standing — pause and bring in the right voices

Practitioner Checklist

• Have you identified all affected stakeholders, including non-users?
• Is the decision grounded in a recognized framework (NIST, ISO, EU AI Act, internal)?
• Have you measured the relevant fairness, privacy, transparency, and safety metrics?
• Is there a documented decision record (ADR) with the reasoning, dissent, and alternatives?
• Is there a plan to monitor real-world impact and re-evaluate?
• Have you involved the right voices (legal, ethics, impacted communities, regulators where appropriate)?

Next Steps

The other lessons in Algorithmic Fairness Metrics build directly on this one. Once you are comfortable with predictive parity, the natural next step is to combine it with the patterns in the surrounding lessons — that is where ethical practice goes from one-off decisions to an operating system. Ethics is most useful as a system, not as isolated reviews.