Metrics & Measurement

The north star metric should capture the ultimate user and business value, not just model performance. For content moderation, I would define it as:

North Star: "Percentage of users who report feeling safe on the platform" (measured via quarterly survey)

This is the right north star because it captures the outcome we care about, not the mechanism. Below it, I would build a metrics hierarchy:

• Model metrics: Precision (avoid removing legitimate content), recall (catch harmful content), latency (moderate before content spreads)
• Product metrics: Time-to-action (how quickly harmful content is removed), false positive rate experienced by creators, appeal success rate
• Business metrics: User retention, advertiser confidence scores, regulatory compliance rate
• Guardrail metrics: Over-moderation rate (legitimate content wrongly removed), demographic parity in moderation decisions, creator satisfaction

Why not just use model accuracy? Because a model with 99% accuracy might still miss the 1% of content that is most harmful (targeted harassment, CSAM). And a model that is too aggressive removes legitimate speech and drives away creators. The north star forces us to optimize for the right balance.

A/B testing AI features is harder than testing traditional features for three reasons: (1) AI quality varies by input, so aggregate metrics can hide segment-level problems, (2) user behavior changes over time as they learn to trust or distrust the AI, and (3) novelty effects inflate short-term engagement.

My approach to AI A/B testing:

• Longer test duration: Run for at least 4 weeks instead of the typical 2. Users need time to calibrate trust with AI features. Week 1 data is often misleading due to novelty.
• Segment analysis: Do not just look at aggregate results. Slice by query difficulty, user expertise, and input type. An AI feature might help beginners but annoy experts, and the aggregate looks flat.
• Multiple metrics: Track both immediate (click-through, acceptance rate) and delayed (7-day retention, repeat usage, task completion) metrics. AI features that boost short-term engagement can hurt long-term trust.
• Guardrail metrics in the test: Set kill criteria before launch. If the AI feature causes a 5% increase in support tickets, a 10% increase in user-reported errors, or any safety incident, stop the test.
• Interleaving for ranking systems: For search and recommendations, use interleaving instead of split testing. Show results from both control and treatment in the same result page and measure which results users prefer.

Common pitfall: Testing AI on/off rather than testing AI quality levels. Sometimes the right comparison is "AI at 85% accuracy" vs "AI at 90% accuracy," not "AI" vs "no AI."

Guardrail metrics are metrics that must not degrade when you optimize for your primary metric. They prevent you from achieving your goals at an unacceptable cost.

Why AI products need guardrails more than traditional products:

• AI optimization is powerful and can find shortcuts that technically improve the target metric while causing harm. A recommendation system optimized for engagement might promote outrage content.
• AI features affect different user groups differently. Without guardrails, improvements for the majority can come at the expense of minorities.
• Model behavior is harder to predict than code behavior. Guardrails act as safety nets for unexpected model outputs.

Essential guardrail categories for AI products:

Key principle: Define guardrails before you launch, not after something goes wrong. Every AI feature should have at least 3 guardrail metrics with explicit thresholds and escalation procedures.

Model accuracy improvements do not translate linearly to business impact. A 5 percentage point improvement can be transformative or negligible depending on context. Here is how I would measure and communicate the impact:

Step 1 — Quantify error reduction: Going from 90% to 95% means the error rate dropped from 10% to 5% — that is a 50% reduction in errors. Frame it as error reduction, not accuracy gain, because errors are what users experience.

Step 2 — Map errors to business costs:

• How many errors occurred per day at 90% accuracy? If 100K predictions/day, that is 10K errors reduced to 5K.
• What is the cost per error? Support ticket ($5), user churn (LTV of $200), or refund ($50)?
• Total impact: 5K fewer errors/day × cost per error = daily savings.

Step 3 — Consider non-linear effects:

• Trust thresholds: There may be a threshold (e.g., 93%) below which users do not trust the feature and above which they adopt it. The jump from 90% to 95% might cross this threshold, causing a step-function increase in adoption.
• Edge case distribution: Are the remaining 5% errors concentrated in high-value users or high-stakes scenarios? Fixing errors for premium users may have outsized business impact.
• Competitive parity: If competitors are at 94%, going from 90% to 95% might flip competitive win rates.

Step 4 — Measure, do not estimate: A/B test the new model. Measure actual changes in user behavior, support volume, and revenue — do not rely on theoretical calculations alone.

This is one of the most common and frustrating situations in AI product development. There are several likely causes:

• Train-serve skew: The data distribution in production is different from the test set. The model improved on test data but production data has different patterns, edge cases, or data quality.
• Offline metric misalignment: The offline metric (e.g., accuracy on a held-out set) does not correlate with the online metric (e.g., user satisfaction). A model that is more accurate at easy cases but worse at hard cases might show offline gains that users never notice.
• Positional bias: In ranking systems, improvements in lower-ranked items may not matter because users rarely scroll that far.
• User adaptation: Users have already adapted their behavior to the old model. The new model might be objectively better but users have learned workarounds that mask the improvement.
• Feature not the bottleneck: The AI model is not the limiting factor in the user experience. If the UI is confusing or the latency is high, a better model does not help.

How to investigate: Segment online results by the cases where the new model differs from the old model. If the new model changed predictions for 1,000 users, what happened to those specific users? This isolates the impact of the model change from overall trends.

I would organize metrics into a hierarchy that connects model quality to user value to business outcomes:

Level 1 — Model quality:

• Suggestion relevance (human-rated quality of top suggestion)
• Latency (time from keystroke to suggestion appearing)
• Factual accuracy rate (for factual claims in generated text)

Level 2 — User engagement:

• Acceptance rate: percentage of suggestions users accept (target: 25–35%)
• Edit rate: how much users modify accepted suggestions (lower is better)
• Time saved per document: writing time with AI vs without (baseline comparison)
• Feature retention: percentage of users still using the feature after 30 days

Level 3 — Business impact:

• User acquisition: conversion rate uplift from AI feature in marketing
• Retention: 30/60/90 day retention for AI users vs non-AI users
• Revenue: willingness to pay (for premium AI features), upsell conversion

Guardrails:

• Distraction rate: percentage of users who report suggestions are distracting (must stay below 15%)
• Over-reliance: users should not accept every suggestion blindly (acceptance rate above 80% is a red flag)
• Homogenization: diversity of writing styles across users should not decrease (monitor with style entropy metrics)

Trust is the single most important and hardest-to-measure metric for AI products. I use a combination of behavioral and attitudinal signals:

Behavioral signals (implicit trust measurement):

• Adoption curve: How quickly do users go from trying to regularly using the AI feature? A steep curve suggests rapid trust-building.
• Override rate over time: If users initially override 40% of AI suggestions but this drops to 15% over 4 weeks, trust is building. If it increases, trust is eroding.
• Automation level increase: In products with adjustable AI autonomy, are users increasing the AI's authority over time?
• Recovery behavior: After the AI makes a mistake, do users give it another chance or disable the feature? Fast recovery indicates resilient trust.
• Verification behavior: Do users double-check AI outputs? Decreasing verification over time signals growing trust.

Attitudinal signals (explicit trust measurement):

• In-product surveys: "How confident are you in this AI suggestion?" on a 1–5 scale, triggered periodically.
• NPS delta: Compare NPS scores of AI feature users vs non-users. Positive delta means AI is adding trust, not eroding it.
• Willingness to delegate: "Would you trust this AI feature with [higher-stakes task]?" measures trust ceiling.

Trust index: Combine 3 behavioral + 2 attitudinal signals into a composite trust score. Track weekly. Alert if it drops more than 10% in any segment.

The cold start problem creates a measurement challenge: your AI feature needs data to perform well, but you need it to perform well to collect data. This creates a chicken-and-egg problem for metrics.

Strategies:

• Separate v1 metrics from steady-state metrics: Your launch metrics should be different from your long-term metrics. At launch, measure data collection rate, user onboarding completion, and initial engagement. Save accuracy-dependent metrics for 30 days post-launch.
• Use leading indicators: Instead of measuring recommendation quality (which requires user history), measure session depth, browse time, and explicit preference signals that indicate users are engaging enough to generate data.
• Benchmark against heuristics: Compare AI performance against simple rules-based alternatives. At launch, the AI might not beat heuristics — that is okay. Track the crossover point where AI surpasses heuristics as a milestone.
• Cohort analysis: Track metric improvement over user tenure. "Users in their 4th week have 2x higher satisfaction than users in their 1st week" shows the AI is learning and improving.
• Synthetic benchmarks: Use held-out evaluation sets to measure model quality independently of user behavior. This lets you prove the model is improving even before user-facing metrics catch up.

This is a common question that is deceptively hard because AI investments often have indirect and long-term returns. Here is my framework:

Direct ROI (measurable today):

• Cost savings: AI automation replacing manual processes. "Our AI moderation system handles 80% of content reviews, saving $2M/year in human moderator costs."
• Revenue increase: AI features driving engagement or conversion. "AI-powered recommendations increased average order value by 12%, adding $5M in annual revenue."
• Efficiency gains: "AI-assisted search reduced customer time-to-resolution by 35%, increasing customer satisfaction by 15 NPS points."

Indirect ROI (measurable over 6–12 months):

• Competitive positioning: AI features as a differentiator in sales conversations. "40% of enterprise deals cite our AI capabilities as a key decision factor."
• Data flywheel: Each AI feature generates data that makes future AI features better and cheaper to build.
• Talent attraction: AI capabilities attract top engineering talent, reducing recruiting costs and time-to-hire.

What to avoid: Do not inflate numbers to justify the investment. If the ROI is unclear, say "We are in the investment phase. Here is how we will measure ROI in 6 months, and here are the early signals that suggest we are on track." VPs respect honesty more than inflated projections.

Metric trade-offs are inevitable in AI products. The classic examples: precision vs recall, engagement vs quality, speed vs accuracy. Here is my decision framework:

Step 1 — Make the trade-off explicit: Quantify the relationship. "Improving recall from 80% to 90% reduces precision from 95% to 88%." Never make the trade-off without knowing the exact numbers.

Step 2 — Map to user impact: Translate the metric trade-off into user stories. "Higher recall means we catch 10% more harmful content, but 7% more legitimate posts get flagged for review." This makes the trade-off concrete and debatable.

Step 3 — Identify the constraint: Which metric has a hard floor? In content moderation, recall below 85% is unacceptable (missing harmful content). In search, precision below 80% is unacceptable (bad results erode trust). The constrained metric becomes the guardrail; optimize the other.

Step 4 — Look for Pareto improvements: Before accepting the trade-off, ask: can we improve both metrics simultaneously? Better training data, a different model architecture, or a two-stage system (high-recall first stage, high-precision second stage) might eliminate the trade-off.

Step 5 — Decide and document: Make the decision, write down the reasoning, and set a review date. "We chose to optimize for recall with a precision floor of 90%. We will revisit in Q3 when we have more training data."