LlamaIndex User Guide

LlamaIndex is an open-source data framework for building LLM and agentic applications in Python. It connects your private data — PDFs, documents, APIs, SQL databases, and more than 300 other sources — to large language models so you can ask questions over it, build retrieval-augmented generation (RAG) pipelines, and run autonomous agents grounded in your own content.

This guide shows you how to install LlamaIndex, load your data, build an index, query it, and customize every part of the pipeline using its plugin ecosystem. It is written for application developers who want to use LlamaIndex. If you operate, deploy, or extend the framework, see the companion Admin Guide.

Note

LlamaIndex is a Python library, not a command-line application. You build with it by importing classes into your own Python code. The only shipped command-line tool, llama-dev, is a contributor utility for the source repository (covered in the Admin Guide), not something end users run.

1. Quick Start

If you already have Python 3.10 or newer and an OpenAI API key, you can index a folder of documents and query it in five lines of code.

pip install llama-index

import os

os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("YOUR_DATA_DIRECTORY").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
response = query_engine.query("What is this collection of documents about?")
print(response)

This collection of documents describes ... (an answer synthesized from your files,
grounded in the most relevant passages retrieved from YOUR_DATA_DIRECTORY)

What happened in those lines:

1. SimpleDirectoryReader loaded every supported file in the directory into a list of Document objects.
2. VectorStoreIndex.from_documents split the documents into chunks (nodes), embedded each chunk, and stored the vectors in an in-memory index.
3. as_query_engine() created a query engine that, on each query(), retrieves the most relevant chunks and asks the LLM to answer using them as context.

Tip

The llama-index starter package bundles core plus the OpenAI LLM and embedding integrations, so the snippet above works with no extra installs. To use a different provider (Ollama, Anthropic, Hugging Face, etc.), install that provider's integration package — see Section 7.

2. Installation

LlamaIndex offers two installation paths. Choose based on how much control you want over which providers are pulled in.

2.1 Starter install (recommended for getting started)

The umbrella llama-index package includes core plus a curated selection of integrations so you can be productive immediately.

pip install llama-index

This installs:

Important

The starter package defaults to OpenAI for both the LLM and embeddings. You must set the OPENAI_API_KEY environment variable before running, or supply a different provider via Settings (see Section 6).

2.2 Customized install (recommended for production)

Install core alone, then add only the integration packages your application needs. This keeps your dependency tree small and avoids pulling in OpenAI if you don't use it.

pip install llama-index-core
pip install llama-index-llms-ollama
pip install llama-index-embeddings-huggingface
pip install llama-index-vector-stores-qdrant

There are over 300 integration packages on LlamaHub that work seamlessly with core, letting you choose your preferred LLM, embedding model, and vector store providers.

2.3 Requirements

Note

LlamaIndex bundles a small _static folder inside llama-index-core containing cached nltk and tiktoken data, so it can run in environments with restrictive disk access at runtime. These assets are verifiable with GitHub build-provenance attestation; see the Admin Guide's security section.

2.4 Verify your installation

import llama_index.core
print(llama_index.core.__version__)

0.14.22

3. Understanding the Import Convention

LlamaIndex is namespaced so that you can always tell core code from integration code by reading the import.

# typical pattern
from llama_index.core.xxx import ClassABC          # core submodule xxx
from llama_index.xxx.yyy import SubclassABC          # integration yyy for submodule xxx

# concrete example
from llama_index.core.llms import LLM                # the abstract LLM interface (core)
from llama_index.llms.openai import OpenAI           # a concrete LLM (integration)

The rule:

• An import path with core (e.g. llama_index.core.llms) comes from the framework itself.
• An import path without core (e.g. llama_index.llms.openai, llama_index.vector_stores.qdrant) comes from a separately installed integration package.

Tip

If you get a ModuleNotFoundError for an import like llama_index.llms.anthropic, it almost always means you haven't installed that integration. Run pip install llama-index-llms-anthropic.

4. Loading Your Data

Everything in LlamaIndex starts with a Document — a container for a piece of source content plus metadata.

4.1 Load files from a directory

SimpleDirectoryReader is the fastest way to ingest local files. It auto-detects file types and reads every supported file in a folder.

from llama_index.core import SimpleDirectoryReader

documents = SimpleDirectoryReader("./data").load_data()
print(f"Loaded {len(documents)} documents")

Loaded 12 documents

Useful options:

documents = SimpleDirectoryReader(
    input_dir="./data",
    recursive=True,                       # descend into subdirectories
    required_exts=[".pdf", ".md", ".txt"],  # only these extensions
).load_data()

4.2 Create documents manually

When your text comes from somewhere other than files (a database row, an API response, a string), build Document objects directly.

from llama_index.core import Document

doc = Document(
    text="LlamaIndex connects your data to large language models.",
    metadata={"source": "notes", "author": "team"},
)

Metadata is powerful: it is injected into the text shown to the LLM and used for filtering during retrieval. You can control exactly what metadata the embedding model and the LLM each see:

doc.excluded_embed_metadata_keys = ["author"]   # hide from the embedding model
doc.excluded_llm_metadata_keys = ["source"]     # hide from the LLM prompt

4.3 Load from other sources (readers / connectors)

Local files are only the beginning. LlamaIndex ships data connectors (also called readers) for hundreds of sources — Notion, Slack, Google Drive, S3, SQL databases, web pages, GitHub, Confluence, MongoDB, and many more. Each lives in its own integration package.

pip install llama-index-readers-web

from llama_index.readers.web import SimpleWebPageReader

documents = SimpleWebPageReader().load_data(
    ["https://example.com/article"]
)

The reader naming convention is always llama-index-readers-<source>. Browse the full catalog on LlamaHub.

5. Building and Querying an Index

An index is a data structure built over your documents that makes them efficiently searchable by an LLM. The most common is VectorStoreIndex, which uses semantic (embedding) search.

5.1 Build a vector index

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)

When you build the index, LlamaIndex automatically:

• Splits each document into nodes (chunks) — by default about 1024 tokens each with 20 tokens of overlap.
• Generates an embedding vector for each node.
• Stores nodes and vectors for retrieval.

5.2 Query the index

query_engine = index.as_query_engine()
response = query_engine.query("How do I configure authentication?")
print(response)

By default the query engine retrieves the 2 most similar nodes and passes them to the LLM as context. You can change this:

query_engine = index.as_query_engine(similarity_top_k=5)

5.3 Inspect the sources behind an answer

A response object carries the source nodes it was built from, so you can show citations or verify grounding.

response = query_engine.query("What is the refund policy?")

print(response.response)            # the synthesized answer text
for node in response.source_nodes:  # the retrieved chunks
    print(node.score, node.node.metadata.get("file_name"))
    print(node.node.get_content()[:200])

5.4 Chat instead of one-shot Q&A

For multi-turn conversations that remember context, use a chat engine instead of a query engine.

chat_engine = index.as_chat_engine()
print(chat_engine.chat("Summarize the onboarding document."))
print(chat_engine.chat("Now list the action items from it."))

5.5 Retrieve without synthesizing an answer

If you only want the relevant chunks (for example, to feed them into your own prompt), use a retriever directly.

retriever = index.as_retriever(similarity_top_k=3)
nodes = retriever.retrieve("payment failures")
for n in nodes:
    print(n.score, n.get_content()[:100])

5.6 Run queries asynchronously

Every major entry point has an async counterpart prefixed with a (aquery, aretrieve, achat). Use these inside async applications.

response = await query_engine.aquery("What changed in the latest release?")

Warning

Calling a synchronous method like query() from inside an already-running event loop (for example, a Jupyter notebook or an async web handler) can raise a "Detected nested async" error. Use the async entry methods (aquery, aretrieve, achat) in those contexts, or apply nest_asyncio.apply(). See Troubleshooting.

6. Customizing the LLM and Embeddings

By default LlamaIndex uses OpenAI. To use any other provider, install its integration and assign it. There are two scopes: global (applies everywhere) and local (applies to one call).

6.1 Global configuration with Settings

Settings is the global configuration object. Set it once at startup and every index, retriever, and query engine picks it up.

from llama_index.core import Settings, VectorStoreIndex, SimpleDirectoryReader
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.ollama import Ollama
from transformers import AutoTokenizer

# set the LLM (served locally through Ollama)
Settings.llm = Ollama(model="llama-3.1:latest", request_timeout=360.0)

# set the tokenizer to match the LLM
Settings.tokenizer = AutoTokenizer.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct"
)

# set the embedding model (runs locally via Hugging Face)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)

Commonly set fields on Settings:

Tip

Setting the embedding model is the most important customization for cost and privacy. A local Hugging Face embedding model means your documents are never sent to a third-party embedding API.

6.2 Local (per-call) configuration

Pass a model directly to a single call to override the global default just for that operation.

from llama_index.llms.anthropic import Anthropic

query_engine = index.as_query_engine(llm=Anthropic(model="claude-sonnet-4-5"))

Caution

Older tutorials use ServiceContext to bundle these settings. ServiceContext is deprecated. Use Settings for global configuration, or pass modules directly to functions. Code that constructs a ServiceContext will raise an error pointing to the migration guide.

7. The Integration Ecosystem

LlamaIndex's power comes from its plugin model: a small, stable core plus 300+ swappable integration packages. Every integration implements an abstract base class from core, so you can change providers without rewriting your application logic.

7.1 Categories of integrations

7.2 The swap-a-provider pattern

Because every provider implements the same interface, switching is a one-line change. To move from OpenAI to Anthropic:

# before
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o")

# after — install llama-index-llms-anthropic, then:
from llama_index.llms.anthropic import Anthropic
Settings.llm = Anthropic(model="claude-sonnet-4-5")

The same applies to embeddings, vector stores, and readers. Your indexing and querying code never changes.

7.3 Use a persistent vector store

For anything beyond prototyping, store vectors in a real database instead of memory. The pattern is: create the store, wrap it in a StorageContext, and pass that when building the index.

pip install llama-index-vector-stores-qdrant

import qdrant_client
from llama_index.core import VectorStoreIndex, StorageContext, SimpleDirectoryReader
from llama_index.vector_stores.qdrant import QdrantVectorStore

client = qdrant_client.QdrantClient(path="./qdrant_data")
vector_store = QdrantVectorStore(client=client, collection_name="my_docs")
storage_context = StorageContext.from_defaults(vector_store=vector_store)

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)

8. Persisting and Reloading an Index

By default, an in-memory VectorStoreIndex is lost when your program exits. You can persist it to disk and reload it later without re-embedding.

8.1 Persist to disk

index.storage_context.persist()              # writes to ./storage by default
index.storage_context.persist(persist_dir="./my_storage")

8.2 Reload from disk

from llama_index.core import StorageContext, load_index_from_storage

storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context)

Important

When reloading, make sure your Settings.embed_model matches the model used to build the index. Reloading with a different embedding model produces meaningless similarity scores. The LLM can differ; the embedding model should not.

Note

Disk persistence is for local file-based storage. When you use an external vector store (Qdrant, Pinecone, etc.), the vectors already live in that database — you reconnect to it instead of loading from ./storage.

9. Choosing the Right Index

VectorStoreIndex covers most use cases, but LlamaIndex offers several index types for different retrieval needs. All are importable from llama_index.core.

from llama_index.core import SummaryIndex

summary_index = SummaryIndex.from_documents(documents)
response = summary_index.as_query_engine().query(
    "Give me a one-paragraph summary of all the documents."
)

Tip

Start with VectorStoreIndex. Reach for SummaryIndex only when you genuinely need to consider every node (full-corpus summaries), and for graph-style questions about entities and relationships, use PropertyGraphIndex.

10. Troubleshooting

These errors come directly from LlamaIndex's source. Each entry gives the message, cause, and fix.

Error: Detected nested async. Please use nest_asyncio.apply() to allow nested event loops.

• Cause: You called a synchronous method (e.g. query()) from inside an already-running event loop — common in Jupyter notebooks and async web frameworks.
• Solution: Use the async methods instead (aquery, aretrieve, achat), or call import nest_asyncio; nest_asyncio.apply() once at startup.
• Prevention: In async codebases, default to the a-prefixed methods everywhere.

Error: ServiceContext is deprecated. Use llama_index.settings.Settings instead...

• Cause: Your code (or an old tutorial you copied) constructs a ServiceContext.
• Solution: Replace it with Settings for global config (Settings.llm = ..., Settings.embed_model = ...), or pass llm=/embed_model= directly to the call.
• Prevention: Follow current docs; treat any ServiceContext reference as outdated.

Error: ModuleNotFoundError: No module named 'llama_index.llms.<provider>'

• Cause: The integration package isn't installed. Core does not bundle every provider.
• Solution: pip install llama-index-llms-<provider> (or -embeddings-, -vector-stores-, -readers- for other categories).
• Prevention: Remember the import rule — any import without core requires a separately installed package.

Error: embedding not set.

• Cause: You tried to read .get_embedding() on a node that was never embedded.
• Solution: Build the node through an index (which embeds it) or set Settings.embed_model before indexing.
• Prevention: Don't access embeddings on raw Document/TextNode objects you created by hand.

Error: Configured node parser does not have chunk size. / ...chunk overlap.

• Cause: You set a custom node_parser on Settings and then read Settings.chunk_size/chunk_overlap, but that parser doesn't expose those attributes.
• Solution: Configure chunk size/overlap on the parser itself, or use a sentence-splitter-style parser that supports them.
• Prevention: Set chunk_size/chunk_overlap or a full custom node_parser, not both inconsistently.

Error: Source object must be a single RelatedNodeInfo object (also Previous/Next/Parent/Child variants)

• Cause: A node's relationships were set with the wrong shape — e.g. a list where a single relation is expected, or vice versa for child nodes.
• Solution: Set SOURCE/PREVIOUS/NEXT/PARENT to a single RelatedNodeInfo, and CHILD to a list of them.
• Prevention: Use node.as_related_node_info() to build relationship values correctly.

Authentication / rate-limit failures from your provider

• Cause: Missing API key (e.g. OPENAI_API_KEY not set) or hitting your provider's request/token limits.
• Solution: Set the required environment variable for your provider. For rate limits, attach a rate limiter (see the Admin Guide) or lower concurrency.
• Prevention: Verify the key is set before your first call; budget for provider limits in batch jobs.

Tip

For community-reported issues and their resolutions, search the project's GitHub issue tracker. Recent examples include obj-serialization mismatches and refresh_ref_docs() dropping keyword arguments after the first document in a batch — both useful to know if you build incremental ingestion pipelines.

11. Frequently Asked Questions

Q: What is the difference between llama-index and llama-index-core? llama-index is a starter bundle (core + OpenAI LLM/embeddings + nltk). llama-index-core is just the framework; you add the integration packages you want on top of it.

Q: Do I have to use OpenAI? No. OpenAI is only the default in the starter package. Install any LLM/embedding integration and assign it via Settings to use Ollama, Anthropic, Hugging Face, Bedrock, Mistral, and many others.

Q: Can I run everything locally with no external API calls? Yes. Use a local LLM (e.g. Ollama, vLLM) and a local embedding model (e.g. Hugging Face) and a local vector store. Your data then never leaves your machine.

Q: How large can my data be? LlamaIndex scales by swapping the in-memory index for an external vector store (Qdrant, Pinecone, Milvus, Postgres/pgvector, etc.). The framework itself doesn't impose a hard limit; your vector store and embedding throughput do.

Q: How do I cite or verify the sources of an answer? Read response.source_nodes — each carries the retrieved text, its metadata (such as file_name), and a relevance score.

Q: My answers ignore some documents. Why? By default only the top 2 most similar chunks are retrieved. Increase similarity_top_k, or add a reranker postprocessor to improve which chunks reach the LLM.

Q: How do I do multi-turn chat? Use index.as_chat_engine() instead of as_query_engine(). The chat engine maintains conversation history across turns.

Q: What's the difference between a retriever, a query engine, and a chat engine? A retriever returns relevant chunks. A query engine retrieves and asks the LLM to synthesize an answer (one-shot). A chat engine does the same but remembers conversation history.

Q: Is LlamaIndex safe to expose directly to end users on the internet? LlamaIndex is designed to run inside a trusted environment (backend code, jobs, scripts). If you put it behind a public endpoint, input validation, authentication, and rate limiting are your responsibility — see the Admin Guide's security section.

Q: What is LlamaParse / LlamaCloud? A separate commercial platform from the same company for agentic document parsing, extraction, and indexing. It works with this open-source framework but is not part of it.

Q: How do I build an agent that can call tools? Install llama-index-tools-* packages for the capabilities you want and compose them with LlamaIndex's agent and Workflows abstractions. See the framework documentation for agent tutorials.

12. Quick Reference

Install

pip install llama-index                      # starter (core + OpenAI + nltk)
pip install llama-index-core                 # core only
pip install llama-index-llms-<provider>      # add an LLM
pip install llama-index-embeddings-<provider> # add embeddings
pip install llama-index-vector-stores-<db>   # add a vector store
pip install llama-index-readers-<source>     # add a data connector

Minimal RAG pipeline

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
print(index.as_query_engine().query("..."))

Common defaults (from core)

Key imports from llama_index.core

from llama_index.core import (
    Document, Settings, StorageContext,
    VectorStoreIndex, SummaryIndex, TreeIndex,
    KeywordTableIndex, KnowledgeGraphIndex, PropertyGraphIndex,
    DocumentSummaryIndex, SimpleDirectoryReader,
    load_index_from_storage,
)

Async entry points