Intermediate

Indexing

Learn about LlamaIndex's index types, node parsers, text splitters, and how to choose the right indexing strategy for your data and use case.

Index Types

LlamaIndex provides several index types, each optimized for different query patterns:

📦

VectorStoreIndex

Embeds chunks as vectors for semantic similarity search. The most common index type, ideal for RAG and Q&A.

📝

SummaryIndex

Reads through all nodes sequentially. Best for summarization tasks where you need to process entire documents.

📈

KnowledgeGraphIndex

Extracts entities and relationships into a knowledge graph. Best for structured data and relationship queries.

🌳

TreeIndex

Builds a tree of summaries from leaf nodes up. Good for multi-level summarization and hierarchical data.

VectorStoreIndex (Most Common)

Python

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

# Load and index documents
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)

# Persist to disk (avoid re-embedding)
index.storage_context.persist(persist_dir="./storage")

# Reload from disk
from llama_index.core import StorageContext, load_index_from_storage
storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context)

Node Parsers

Node parsers control how documents are split into chunks (nodes). The default is SentenceSplitter:

Python

from llama_index.core.node_parser import (
    SentenceSplitter,
    TokenTextSplitter,
    SemanticSplitterNodeParser
)
from llama_index.core import Settings

# Sentence-based splitting (default)
Settings.node_parser = SentenceSplitter(
    chunk_size=1024,
    chunk_overlap=200
)

# Token-based splitting
Settings.node_parser = TokenTextSplitter(
    chunk_size=1024,
    chunk_overlap=200
)

# Semantic splitting (groups semantically similar sentences)
Settings.node_parser = SemanticSplitterNodeParser(
    embed_model=Settings.embed_model,
    breakpoint_percentile_threshold=95
)

Using External Vector Stores

Python - Pinecone Integration

from pinecone import Pinecone
from llama_index.vector_stores.pinecone import PineconeVectorStore
from llama_index.core import VectorStoreIndex, StorageContext

# Connect to Pinecone
pc = Pinecone(api_key="your-key")
pinecone_index = pc.Index("my-index")
vector_store = PineconeVectorStore(pinecone_index=pinecone_index)

# Create index with Pinecone as backend
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents,
    storage_context=storage_context
)

SummaryIndex

Python

from llama_index.core import SummaryIndex

# SummaryIndex reads all nodes - good for summarization
index = SummaryIndex.from_documents(documents)
query_engine = index.as_query_engine(
    response_mode="tree_summarize"
)
response = query_engine.query("Summarize the main themes")

KnowledgeGraphIndex

Python

from llama_index.core import KnowledgeGraphIndex

# Extract entities and relationships
kg_index = KnowledgeGraphIndex.from_documents(
    documents,
    max_triplets_per_chunk=10
)

# Query relationships
query_engine = kg_index.as_query_engine()
response = query_engine.query("How is entity A related to entity B?")

✅

Choosing an index: Use VectorStoreIndex for 90% of use cases (Q&A, search, RAG). Use SummaryIndex when you need to process entire documents. Use KnowledgeGraphIndex for relationship-heavy data like company structures or research papers.

⚠

Persist your index: Always persist the index to disk after creation. Re-embedding documents is expensive and slow. Use index.storage_context.persist() and reload with load_index_from_storage().

What's Next?

In the next lesson, we will learn how to query indexes — query engines, response synthesis modes, chat engines, and streaming.

← PreviousInstallation & Setup Next →Querying