Advanced

Mosaic AI

Build enterprise generative AI applications using Mosaic AI's foundation model training, agent framework, vector search, and model serving capabilities.

What is Mosaic AI?

Mosaic AI (formerly MosaicML) is Databricks' suite of generative AI tools that enables enterprises to train, fine-tune, and deploy large language models and build compound AI systems. It integrates deeply with the Lakehouse platform for governed, data-aware AI applications.

💡
Key advantage: Unlike third-party LLM APIs, Mosaic AI lets you train custom models on your own data within your cloud environment, ensuring data privacy, compliance, and domain-specific performance.

Mosaic AI Components

💻

Model Training

Pre-train or fine-tune foundation models on your enterprise data using optimized distributed training infrastructure.

🛠

Agent Framework

Build, deploy, and evaluate compound AI agents that combine LLMs with tools, retrieval, and business logic.

📈

Vector Search

Managed vector database for similarity search, powering RAG applications with automatic index management.

⚡

AI Gateway

Unified gateway for routing, rate limiting, and governing access to both internal and external LLM endpoints.

Building RAG Applications

Retrieval-Augmented Generation (RAG) is a core pattern for enterprise AI:

RAG with Mosaic AI
# 1. Create a Vector Search index
from databricks.vector_search.client import VectorSearchClient

vsc = VectorSearchClient()
index = vsc.create_delta_sync_index(
    endpoint_name="vs_endpoint",
    source_table_name="catalog.schema.documents",
    index_name="catalog.schema.doc_index",
    pipeline_type="TRIGGERED",
    embedding_source_column="content",
    embedding_model_endpoint_name="databricks-bge-large-en"
)

# 2. Query the index for relevant documents
results = index.similarity_search(
    query_text="What is our return policy?",
    columns=["content", "source"],
    num_results=5
)

AI Agent Framework

Build production-ready AI agents with the Mosaic AI Agent Framework:

  • Agent authoring: Define agents using LangChain, LlamaIndex, or custom Python code
  • Tool integration: Connect agents to Unity Catalog functions, SQL queries, and external APIs
  • Agent evaluation: Automated evaluation with LLM judges for quality, safety, and accuracy
  • Deployment: Deploy agents as serverless endpoints with built-in monitoring and guardrails
  • Review app: Stakeholder feedback collection with a built-in review application

AI Gateway

The AI Gateway provides centralized governance for LLM access:

  • Unified endpoint: Single API for accessing Databricks-hosted models, external APIs (OpenAI, Anthropic), and custom models
  • Rate limiting: Per-user and per-application rate limits to control costs
  • Guardrails: Content filtering, PII detection, and safety checks on inputs and outputs
  • Usage tracking: Detailed logs of all LLM interactions for auditing and cost management
✅
Key takeaway: Mosaic AI enables enterprises to build custom generative AI applications with full control over their data and models. The combination of vector search, agent framework, and AI gateway provides a complete stack for production AI systems.