Lilly Tech Systems
Intermediate
Code Workbook
Use Foundry's interactive development environment for data exploration, transformation prototyping, and ML model building with Python, SQL, and R.
What is Code Workbook?
Code Workbook is Foundry's interactive notebook-like environment for exploratory analysis and rapid development. It combines the interactivity of Jupyter notebooks with Foundry's governance, data access, and pipeline integration.
Code Workbook vs. Code Repositories: Code Workbook is for interactive exploration and prototyping. Code Repositories are for production-grade, version-controlled pipeline code. Many teams prototype in Workbook and promote to Repositories.
Languages & Features
| Language | Engine | Best For |
|---|---|---|
| Python | PySpark / pandas | Data transforms, ML models, general-purpose analysis |
| SQL | SparkSQL | Data querying, aggregations, quick analysis |
| R | SparkR | Statistical analysis, specialized R packages |
Key Capabilities
- DAG visualization: See the dependency graph of your transforms as a visual pipeline
- Data previews: Preview intermediate results at any step without running the full pipeline
- Environment management: Install Python packages from Foundry's managed conda channels
- Scheduling: Promote workbook transforms to scheduled pipelines with one click
- Collaboration: Share workbooks with team members and leave comments on code cells
- GPU support: Access GPU compute for deep learning model training
ML Model Building
Code Workbook supports the full ML lifecycle:
- Feature engineering: Build features using Foundry datasets with full governance
- Model training: Use scikit-learn, XGBoost, PyTorch, TensorFlow with managed compute
- Model registration: Register trained models in Foundry's model catalog
- Model deployment: Deploy models as Ontology functions for real-time inference
- Monitoring: Track model performance metrics over time with automated alerts
Code Workbook vs. Alternatives
| Feature | Code Workbook | Jupyter Notebook |
|---|---|---|
| Data governance | Built-in Foundry governance | Manual / external |
| Pipeline integration | Direct promotion to production | Manual export required |
| Compute management | Managed Spark clusters | Self-managed |
| Collaboration | Native sharing with RBAC | File-based sharing |
| Dependency tracking | Automatic DAG | Manual cell ordering |
Key takeaway: Code Workbook bridges the gap between ad-hoc exploration and production pipelines. Its integration with Foundry's governance and pipeline system means work done in Workbook can be promoted to production without rewriting code.