Performance Monitoring

Monitor AI vendor performance continuously rather than only when there is a problem. Learn the metric set (uptime, latency p50/p95/p99, model quality on a held-out eval set, error rate, drift on canary inputs), instrumentation that does not depend on vendor self-reporting, dashboards that the vendor manager and the business owner share, anomaly detection, and the escalation playbook when metrics breach thresholds.