Lilly Tech Systems
Advanced
Foundry Best Practices
Enterprise-proven patterns for Ontology design, data governance, pipeline architecture, and production deployment on Palantir Foundry.
Ontology Design Patterns
- Start with use cases: Design object types around operational decisions, not data schemas
- Keep objects meaningful: Each object type should represent a real-world entity stakeholders recognize
- Use interfaces: Define shared contracts (e.g., "Locatable", "Timestamped") for consistent behavior
- Limit link depth: Avoid deeply nested link chains; flatten where possible for performance
- Version your Ontology: Treat Ontology changes like schema migrations with review processes
Data Pipeline Best Practices
| Practice | Recommendation |
|---|---|
| Incremental processing | Use incremental transforms to process only new/changed data |
| Branching | Test pipeline changes on branches before merging to production |
| Data expectations | Add quality checks (null rates, value ranges, row counts) on every output |
| Modular design | Break large pipelines into reusable, composable transforms |
| Documentation | Document every dataset with descriptions, owners, and SLAs |
Security & Governance
- Marking-based security: Use markings to classify data sensitivity and control access automatically
- Project structure: Organize projects by business domain with clear ownership and access policies
- Least privilege: Grant minimum necessary access; use groups rather than individual permissions
- Audit regularly: Review access patterns and unused permissions quarterly
- AIP governance: Define allowed actions and model access per role; log all AI interactions
Production Deployment
Environment Strategy
Maintain separate development and production spaces with promotion workflows and approval gates.
Monitoring
Set up health checks, SLA monitoring, and alerts for pipeline failures and data quality issues.
Change Management
Use code review, branch-based testing, and staged rollouts for Ontology and pipeline changes.
Scalability
Design for incremental processing, partition large datasets, and use appropriate compute profiles.
Key takeaway: Success with Foundry requires treating it as an enterprise platform, not just a tool. Invest in Ontology design, enforce governance from day one, build modular pipelines, and establish clear promotion workflows from development to production.