Lilly Tech Systems
Advanced
API Security for AI Systems
AI inference APIs are the primary interface through which users and applications interact with models. Securing these endpoints requires authentication, authorization, rate limiting, input validation, and output filtering.
Authentication for AI APIs
- API keys: Simple but limited; suitable for service-to-service communication within trusted networks
- OAuth 2.0 / OIDC: Standard for user-facing applications; supports scopes and token-based access
- Mutual TLS (mTLS): Certificate-based authentication for high-security AI endpoints
- Service mesh identity: Use service mesh (Istio, Linkerd) for zero-trust service-to-service AI API authentication
Authorization Patterns
| Pattern | Description | AI Use Case |
|---|---|---|
| Scope-based | OAuth scopes limit what API operations are allowed | Separate read-model vs invoke-model scopes |
| Model-level | Different permissions per model | Restrict access to specialized or sensitive models |
| Tenant isolation | Multi-tenant access boundaries | Each tenant can only access their own models and data |
| Feature gating | Control access to specific AI capabilities | Restrict function calling, code execution, or web access |
Rate Limiting and Quotas
AI APIs are expensive to serve. Implement multi-dimensional rate limiting:
- Requests per minute/hour: Prevent abuse and control costs
- Tokens per minute: Limit the volume of AI processing (input + output tokens)
- Concurrent requests: Prevent resource exhaustion on GPU infrastructure
- Cost-based quotas: Set spending limits per team, project, or application
Input Validation and Output Filtering
- Prompt injection detection: Scan inputs for prompt injection attempts before forwarding to models
- PII detection: Scan inputs for sensitive data and either block, mask, or log the request
- Output scanning: Check model outputs for sensitive information, harmful content, or policy violations
- Content filtering: Apply organization-specific content policies to AI-generated responses
AI Gateway pattern: Deploy an AI gateway (like an API gateway specialized for AI) that handles authentication, rate limiting, input/output filtering, logging, and routing to multiple model backends. This centralizes security controls for all AI API traffic.
Monitoring and Audit
- Log every API request with user identity, model accessed, input/output token counts, and latency
- Monitor for unusual usage patterns that may indicate credential compromise or data exfiltration
- Set alerts for rate limit breaches, authentication failures, and policy violations
- Retain API logs for compliance and forensic investigation purposes
Zero trust for AI APIs: Never trust the network alone. Every AI API request should be authenticated, authorized, and validated regardless of where it originates — even within your internal network.