Quivr Admin & Architecture Guide

This guide is the technical reference for quivr-core — the open-source RAG library that powers Quivr. It is written for the engineers and architects who will embed Quivr in a product, operate it in production, extend it with custom processors or vector stores, or audit its security and internals. For task-oriented usage, see the companion User Guide.

1. Executive Summary

quivr-core is a Python library (package quivr-core, version 0.0.33 at the documented commit) that implements an opinionated Retrieval-Augmented Generation pipeline on top of LangChain and LangGraph. It is not a service, a database, or a UI — it is a dependency you add to your own application.

The repository was historically a full-stack "second brain" web application; it has since been refactored down to this single reusable core plus a docs/ documentation site and four runnable examples/. The 39K GitHub stars reflect that history. Today the product surface is the quivr_core Python API.

What it gives you:

• A single Brain facade that ingests files, indexes them, and answers questions.
• Provider-agnostic LLM access (OpenAI, Azure, Anthropic, Mistral, Gemini, Groq, Ollama).
• A pluggable document-processing registry covering ~15 file formats.
• A swappable vector store (FAISS by default, any LangChain VectorStore accepted).
• A declarative, YAML-driven retrieval workflow compiled into a LangGraph state machine, enabling features like dynamic retrieval, web search, tool use, and citations without code changes.
• Built-in observability via Langfuse.

Architectural stance: Quivr is a trusted-execution library, in the same family as LlamaIndex and LangChain. It assumes the hosting application owns authentication, authorization, input validation, and rate limiting. It is a building block, not a security boundary.

2. Architecture

2.1 High-level component diagram

                          ┌──────────────────────────────────────────────┐
   your application ────► │                  Brain (facade)               │
   (from_files / ask)     │  brain/brain.py — orchestrates everything     │
                          └───┬───────────┬───────────┬───────────┬───────┘
                              │           │           │           │
              ┌───────────────▼──┐  ┌─────▼─────┐ ┌───▼────────┐ ┌▼───────────────┐
              │  Storage          │  │ Processor │ │  Embedder  │ │  Vector store   │
              │  StorageBase      │  │ registry  │ │ Embeddings │ │  VectorStore    │
              │  ├ Transparent    │  │ (lazy)    │ │ (OpenAI)   │ │  (FAISS default)│
              │  └ LocalStorage   │  └─────┬─────┘ └────────────┘ └────────┬────────┘
              └───────────────────┘        │                               │
                                           ▼                               ▼
            INGESTION: file ─► parse ─► split (SplitterConfig) ─► embed ─► store
                                                                            │
                                                                            ▼
                          ┌──────────────────────────────────────────────────────┐
   RETRIEVAL per question │   QuivrQARAGLangGraph  (rag/quivr_rag_langgraph.py)    │
                          │   LangGraph StateGraph compiled from WorkflowConfig    │
                          │   nodes: filter_history → rewrite → retrieve →         │
                          │          [tool_routing → run_tool] → generate_rag      │
                          │   LLMEndpoint (multi-supplier) + RerankerConfig +      │
                          │   prompts registry + LLMToolFactory (web search…)      │
                          └───────────────────────────────┬──────────────────────┘
                                                           ▼
                                          ParsedRAGResponse (answer + citations + sources)
                          Observability: Langfuse callback handler (global)

2.2 Two execution phases

Quivr has a clean split between a one-time ingestion phase and a per-question retrieval phase.

Ingestion (Brain.afrom_files → process_files):

1. Each path is wrapped in a QuivrFile (load_qfile) — computes a SHA-1, detects extension via mimetypes.
2. The file is placed in storage (memory or disk).
3. get_processor_class(extension) resolves a processor (lazy import, priority-ordered).
4. The processor reads the file, splits it (RecursiveCharacterTextSplitter, token-based), and emits LangChain Document chunks; each chunk is enriched with metadata (chunk index, language, file SHA-1, processor class).
5. Chunks are embedded and added to the vector store (FAISS built via FAISS.afrom_documents).

Retrieval (Brain.ask_streaming → QuivrQARAGLangGraph.answer_astream):

1. The configured WorkflowConfig is compiled into a LangGraph StateGraph.
2. The graph runs node by node over an AgentState (messages, chat history, tasks, files…).
3. The final (generate_rag) node's tokens are streamed back; the last chunk carries metadata.

2.3 The workflow-as-graph design (the key idea)

The most important architectural decision is that retrieval logic is data, not code. A WorkflowConfig is a list of NodeConfigs, each naming a method on QuivrQARAGLangGraph and declaring its outgoing edges or a conditional_edge. At runtime _build_workflow calls getattr(self, node.name) to bind each node to its implementing method and wires the edges into a LangGraph StateGraph.

Consequences:

• Adding web search, dynamic retrieval, or prompt-editing is a YAML change, not a code change — as long as the node names map to existing methods.
• The set of available node behaviours is fixed by the methods on QuivrQARAGLangGraph (see §6.3). You cannot invent arbitrary node names; an unknown name raises AttributeError at graph build.
• The first node must be START; the validator check_first_node_is_start enforces this.

2.4 Two RAG engines

Note

Brain.ask_streaming accepts a rag_pipeline argument, but the implementation always instantiates QuivrQARAGLangGraph. The argument is effectively vestigial — the LangGraph engine is what runs. Treat QuivrQARAG as deprecated for new work.

3. Requirements

3.1 Runtime

3.2 Core dependencies (core/pyproject.toml)

3.3 Optional external services

4. Configuration Mastery

All configuration is expressed as Pydantic v2 models that subclass QuivrBaseConfig. Every config object can be loaded from YAML via Config.from_yaml(path).

Important

QuivrBaseConfig sets model_config = ConfigDict(extra="forbid"). Unknown keys raise a validation error. This is why a YAML containing max_input_tokens (a key that does not exist) fails — even though the project's own README shows it. Always validate keys against the model fields below; the in-repo core/example_workflows/*.yaml files are the correct reference.

4.1 Configuration object hierarchy

AssistantConfig
├── retrieval_config: RetrievalConfig
│   ├── llm_config: LLMEndpointConfig
│   ├── reranker_config: RerankerConfig
│   └── workflow_config: WorkflowConfig
│       └── nodes: list[NodeConfig]
│           └── conditional_edge: ConditionalEdgeConfig
└── ingestion_config: IngestionConfig
    └── parser_config: ParserConfig
        ├── splitter_config: SplitterConfig
        └── megaparse_config: MegaparseConfig

4.2 RetrievalConfig

4.3 LLMEndpointConfig

Behaviour worth knowing:

• Model-aware capping (set_llm_model_config): Quivr looks up the model in a built-in LLMModelConfig._model_defaults table (per supplier) and lowers max_context_tokens/max_output_tokens to the model's real limits, logging a warning. Values below MIN_CONTEXT_TOKENS/MIN_OUTPUT_TOKENS (both 4096) raise ValueError.
• Supplier inference (set_llm_model): the supplier is derived by startswith matching against the model table (e.g. claude-* → Anthropic).
• Instance caching: LLMEndpoint.from_config caches instances by the config's __hash__, so identical configs reuse the same client.

Supported suppliers (DefaultModelSuppliers): openai, azure, anthropic, meta, mistral, groq, gemini. Each maps to a LangChain chat client in LLMEndpoint.from_config.

4.4 RerankerConfig

Warning

When supplier is set, RerankerConfig validates that <SUPPLIER>_API_KEY exists in the environment at construction time and raises ValueError if missing. A workflow that names a reranker supplier will fail fast without the key.

4.5 SplitterConfig and MegaparseConfig

4.6 WorkflowConfig, NodeConfig, ConditionalEdgeConfig

• WorkflowConfig: name, nodes: list[NodeConfig], available_tools: list[str] (validated against the tool registry — an unknown tool raises a "did you mean…" error via fuzzy matching), plus internal validated_tools / activated_tools.
• NodeConfig: name (must equal a QuivrQARAGLangGraph method, or START/END), edges or conditional_edge, optional tools (instantiated via LLMToolFactory), description.
• ConditionalEdgeConfig: routing_function (a method name like routing_split or tool_routing) and conditions (a list/dict of next-node names, with START/END resolved to LangGraph sentinels).

4.7 Environment variable reference

Note

There is no central env-var scrape and no .env loaded by the library itself. API keys are read on demand via os.getenv inside the config models (set_api_key). Your application is responsible for populating the environment (the examples use python-dotenv). Missing keys are logged as warnings (the key value is never logged).

5. Repository Structure

quivr/
├── core/                         # the quivr-core package
│   ├── quivr_core/
│   │   ├── brain/                # Brain facade + info + (de)serialization + defaults
│   │   ├── llm/                  # LLMEndpoint multi-supplier client + tokenizer cache
│   │   ├── llm_tools/            # tool registry (web search / Tavily, cited_answer)
│   │   ├── processor/            # parser registry + implementations + splitter
│   │   ├── rag/                  # RAG engines, prompts, entities (config/models/chat)
│   │   ├── storage/              # StorageBase + LocalStorage/TransparentStorage
│   │   ├── files/                # QuivrFile abstraction
│   │   ├── language/             # language enum + fasttext detection
│   │   ├── base_config.py        # QuivrBaseConfig (from_yaml, extra=forbid)
│   │   └── config.py             # MegaparseConfig, ParserType, StrategyEnum
│   ├── tests/                    # pytest suite (markers: base/tika/unstructured/slow)
│   ├── example_workflows/        # canonical workflow YAML (correct keys)
│   └── pyproject.toml            # package metadata + deps
├── docs/                         # mkdocs-material documentation site
└── examples/                     # chatbot, chatbot_voice, quivr-whisper, simple_question

Tip

The codebase is organized by concern, and each concern is fronted by an abstract base class (StorageBase, ProcessorBase) or a registry (processor/registry.py, llm_tools/llm_tools.py). To extend Quivr you implement an ABC or register into a registry — you rarely modify existing files.

6. Module Reference

6.1 brain/brain.py — the Brain facade

The orchestrator. Key surface:

process_files(storage, skip_file_error, **kwargs) is the module-level ingestion helper: it iterates storage, resolves a processor per file, and concatenates the resulting chunks.

brain_defaults.py supplies default_llm() (OpenAI gpt-4o), default_embedder() (OpenAIEmbeddings), and build_default_vectordb() (FAISS), each raising an actionable ImportError if the optional dependency is missing.

6.2 llm/llm_endpoint.py — provider abstraction

• LLMEndpoint.from_config(config) builds the right LangChain chat client for the supplier (ChatOpenAI, AzureChatOpenAI, ChatAnthropic, ChatMistralAI, ChatGoogleGenerativeAI, ChatGroq) and caches instances by config hash.
• supports_func_calling() returns False for llama2, ollama3, and test — driving the non-tool answering path.
• count_tokens() uses a per-model HF tokenizer with an LRU-style cache (LLMTokenizer, capped at 50 MB / 5 tokenizers) and a cl100k_base tiktoken fallback.
• info() returns an LLMInfo dataclass (used by print_info).

6.3 rag/quivr_rag_langgraph.py — the workflow engine

QuivrQARAGLangGraph implements every node behaviour as a method. The available nodes/routers:

Supporting structures: AgentState (the LangGraph state TypedDict), UserTasks/UserTaskEntity (multi-task tracking), reduce_rag_context (shrinks context to 85% of the limit by dropping history then least-relevant docs), and (a)invoke_structured_output (Pydantic-typed LLM calls with a JSON-schema-then-fallback strategy).

Note

retrieve_full_documents_context asserts the vector store exposes get_vectors_by_knowledge_id and is documented in-code as "an enterprise only feature." With the default FAISS store this node will fail — only enable it with a compatible store.

6.4 rag/prompts.py — prompt registry

A frozen MappingProxyType of named ChatPromptTemplates built once at import: RAG_ANSWER_PROMPT, CONDENSE_TASK_PROMPT, SPLIT_PROMPT, TOOL_ROUTING_PROMPT, USER_INTENT_PROMPT, UPDATE_PROMPT, CHAT_LLM_PROMPT, DEFAULT_DOCUMENT_PROMPT, and the ZENDESK_* prompts. The default RAG prompt hard-codes the grounding policy ("use ONLY the provided context… do not apologize… don't cite the source id in the answer").

6.5 processor/ — document parsing

• registry.py: maps extensions → processor class paths with lazy import and a min-heap priority so optional/heavy parsers (MegaParse highest priority) are tried first and fall back to lighter ones on ImportError. get_processor_class, register_processor, available_processors are the public API.
• implementations/default.py: _build_processor factory dynamically generates processor classes wrapping LangChain document loaders (CSVLoader, Docx2txtLoader, UnstructuredPDFLoader, PythonLoader, NotebookLoader, etc.).
• implementations/simple_txt_processor.py: dependency-free .txt processor (works with the base install).
• implementations/tika_processor.py: PDF via a Tika HTTP server (3 retries, configurable URL/timeout).
• implementations/megaparse_processor.py: MegaParse over NATS for many formats.
• processor_base.py: ProcessorBase ABC; process_file enriches chunk metadata (chunk index, quivr_core version, detected language, file metadata) and sanitizes null bytes/encoding.
• splitter.py: SplitterConfig (chunk size/overlap).

6.6 storage/, files/, language/

• storage_base.py — StorageBase ABC (nb_files, get_files, upload_file, remove_file, info).
• local_storage.py — LocalStorage (copies or symlinks to a managed dir, de-dupes by SHA-1) and TransparentStorage (in-memory; the default).
• files/file.py — QuivrFile (slots-based) + QuivrFileSerialized (Pydantic) + FileExtension enum (~24 extensions) + load_qfile.
• language/ — a ~170-entry Language enum and detect_language via fastText.

6.7 rag/entities/

• config.py — all the config models (§4).
• models.py — response/IO models: cited_answer (the citation tool schema), ParsedRAGResponse, ParsedRAGChunkResponse, RAGResponseMetadata, QuivrKnowledge, SearchResult, ChatMessage, LangchainMetadata.
• chat.py — ChatHistory (append, iter_pairs, to_list, time-ordered retrieval).

7. Security Analysis

Caution

Pickle deserialization on Brain.load. Restoring a saved brain calls FAISS.load_local(..., allow_dangerous_deserialization=True), which unpickles the index. A maliciously crafted saved brain can execute arbitrary code on load. Only load brains your application produced or fully trusts. Never expose an endpoint that loads a user-supplied brain directory.

7.1 Threat model

Quivr is a library running inside your trusted process. It does not provide authentication, authorization, multi-tenant isolation, or request rate limiting — those are the hosting application's responsibility (the LlamaIndex/LangChain trust model).

7.2 Observability and PII

A global Langfuse CallbackHandler is attached to LLM calls (LangfuseService). If Langfuse env vars are configured, prompts, contexts, and answers are sent to Langfuse for tracing. Disable or self-host Langfuse if your data cannot leave your perimeter.

8. Operations

8.1 Embedding Quivr in a service

Quivr has no server of its own — you run it inside yours (FastAPI, Flask, a worker, etc.). Operational guidance:

• [ ] Run all Quivr operations on the async path inside async web frameworks (afrom_files, aask, ask_streaming) to avoid the event-loop error (§9).
• [ ] Build brains once and persist them with save(); reload with load() to avoid repeated embedding cost and latency.
• [ ] Cache Brain instances per user/session (the quivr-whisper example keeps a dict of brains keyed by session id).
• [ ] Pre-warm tokenizers with LLMTokenizer.preload_tokenizers([...]) at startup if you count tokens on a hot path.
• [ ] Pin quivr-core and the langchain*/langgraph versions — the LangChain ecosystem moves fast and the constraints are narrow (<0.4).

8.2 Persistence layout

brain.save(folder) writes folder/brain_<id>/ containing config.json (a BrainSerialized: id, name, chat history, LLM config, vector-store config, embedder config, storage config) and a vector_store/ FAISS index. Brain.load(folder/brain_<id>) reconstructs from those.

Note

save()/load() currently support only the FAISS vector store, the OpenAI embedder, and TransparentStorage/LocalStorage. Other combinations raise "can't serialize ... for now". For Postgres/pgvector and other stores you must manage persistence yourself.

8.3 Testing the library

# Inside core/, with dev deps installed (rye sync) and a base env:
pytest -m base               # tests needing the `base` extra
pytest -m "not slow"         # skip slow tests

Markers (core/pyproject.toml): base, tika (needs a Tika server), unstructured (needs the extra), slow. A Dockerized run is provided via core/scripts/run_tests.sh (builds Dockerfile.test, runs tox).

8.4 Building the docs site

cd docs
rye sync
rye run docs          # mkdocs serve
rye run build_docs    # mkdocs build --strict

9. Troubleshooting (Operational)

Symptom: RuntimeError: There is no current event loop

Cause: Synchronous wrapper (from_files/ask) invoked where an event loop already runs (ASGI handler, notebook). The wrappers use asyncio.get_event_loop().run_until_complete. (Issue #3650.) Fix: Use afrom_files/aask/ask_streaming. Prevent: Standardize on async in service code.

Symptom: asyncpg / CancelledError / connection-pool warnings

Cause: Async lifecycle issues when using a Postgres-backed vector store outside a properly scoped event loop (issues #3654, #3557). Fix: Ensure DB connections are created and closed within the same running loop; scope the pool to the request/worker lifecycle. Prevent: Manage the event loop explicitly in long-running services.

Symptom: Pydantic ValidationError loading a workflow YAML

Cause: An unknown key (e.g. max_input_tokens) — models use extra="forbid". Fix: Use the documented field names (§4); start from core/example_workflows/*.yaml.

Symptom: ValueError: The API key for supplier '<x>' is not set

Cause: Reranker/LLM supplier configured without its env key (rerankers validate at construction). Fix: Export the named variable before building the config.

Symptom: RuntimeError: can't send parse request to tika server

Cause: No reachable Tika server for PDFs (3 retries exhausted). Fix: Start apache/tika or set TIKA_SERVER_URL; or install an alternative PDF parser.

Symptom: max_context_tokens is too low

Cause: Token limit set below 4096. Fix: Use ≥ 4096 for both context and output tokens.

10. Code Quality & Patterns

Strengths

• Clean separation of concerns behind ABCs (StorageBase, ProcessorBase) and registries — extensible without editing core files.
• Config-as-data: the entire retrieval pipeline is declarative YAML compiled to LangGraph, a genuinely flexible design.
• Async-first with thin sync wrappers; comprehensive Pydantic validation with helpful error messages.
• Provider abstraction isolates six LLM vendors behind one LLMEndpoint.
• A real pytest suite with environment markers (base/tika/unstructured).

Weaknesses / tech debt (from in-code TODO/FIXME and observed behaviour)

• The sync wrappers' get_event_loop().run_until_complete pattern is fragile in modern async runtimes (the #1 reported bug).
• Brain.add_file() is declared but raise NotImplementedError — files can only be added at construction.
• LocalStorage.remove_file / TransparentStorage.remove_file are NotImplementedError.
• save()/load() are hard-wired to FAISS + OpenAI embedder; other stores aren't serializable.
• parse_chunk_response is annotated in-code as "CONVOLUTED LOGIC / redo this."
• The legacy QuivrQARAG engine duplicates retrieval logic now owned by the LangGraph engine.
• The project's own README/docs YAML examples use the invalid max_input_tokens key (contradicting the code and the in-repo example workflows).
• quivr_rag.py notes a lingering dependency on the umbrella langchain package the maintainers want to drop.

11. Suggested Improvements

1. Replace the sync event-loop wrappers with anyio/asyncio.run-safe execution (or document async-only), closing the most common failure mode.
2. Implement Brain.add_file and remove_file so brains are mutable without a full rebuild.
3. Generalize serialization so save()/load() support any vector store and embedder via a registry, mirroring the processor design.
4. Fix the documentation YAML keys (max_input_tokens → max_output_tokens) and add a config-schema doc generated from the Pydantic models.
5. Retire QuivrQARAG or clearly mark it deprecated and remove the vestigial rag_pipeline argument from ask_streaming.
6. Harden untrusted-document ingestion with sandboxing guidance and optional parser timeouts/limits.
7. Make the saved-brain format pickle-free (or signed) to remove the deserialization risk.

12. Contributing

• Repository: github.com/QuivrHQ/quivr; the package lives in core/.
• Tooling (dev deps in core/pyproject.toml): ruff, flake8 + flake8-black, mypy, pre-commit, pytest (+ pytest-asyncio, pytest-xdist, pytest-benchmark).
• Build backend: hatchling; environments managed with rye (lockfiles checked in).
• Run the suite via pytest or the Dockerized core/scripts/run_tests.sh.
• License: Apache 2.0.

Note

Public-issue activity has slowed and the community has asked whether the project is still maintained (issue #3681). For production use, pin a specific quivr-core version and vendor/patch as needed.

13. Quick Reference

Default values at a glance

Essential APIs

from quivr_core import Brain
from quivr_core.llm import LLMEndpoint
from quivr_core.rag.entities.config import (
    RetrievalConfig, LLMEndpointConfig, IngestionConfig,
)

brain = await Brain.afrom_files(name="b", file_paths=[...])   # build
cfg = RetrievalConfig.from_yaml("workflow.yaml")              # configure
async for c in brain.ask_streaming("q", run_id=uuid4(),       # answer
                                   retrieval_config=cfg):
    ...
await brain.save("path"); Brain.load("path/brain_<id>")       # persist