# Quivr Admin & Architecture Guide

This guide is the technical reference for `quivr-core` — the open-source RAG library that powers Quivr. It is written for the engineers and architects who will **embed Quivr in a product**, operate it in production, extend it with custom processors or vector stores, or audit its security and internals. For task-oriented usage, see the companion User Guide.

---

## 1. Executive Summary

`quivr-core` is a Python library (package `quivr-core`, version `0.0.33` at the documented commit) that implements an opinionated Retrieval-Augmented Generation pipeline on top of **LangChain** and **LangGraph**. It is not a service, a database, or a UI — it is a dependency you add to your own application.

The repository was historically a full-stack "second brain" web application; it has since been refactored down to this single reusable core plus a `docs/` documentation site and four runnable `examples/`. The 39K GitHub stars reflect that history. Today the product surface is the `quivr_core` Python API.

**What it gives you:**

- A single `Brain` facade that ingests files, indexes them, and answers questions.
- Provider-agnostic LLM access (OpenAI, Azure, Anthropic, Mistral, Gemini, Groq, Ollama).
- A pluggable document-processing registry covering ~15 file formats.
- A swappable vector store (FAISS by default, any LangChain `VectorStore` accepted).
- A **declarative, YAML-driven retrieval workflow** compiled into a LangGraph state machine, enabling features like dynamic retrieval, web search, tool use, and citations without code changes.
- Built-in observability via Langfuse.

**Architectural stance:** Quivr is a *trusted-execution library*, in the same family as LlamaIndex and LangChain. It assumes the hosting application owns authentication, authorization, input validation, and rate limiting. It is a building block, not a security boundary.

---

## 2. Architecture

### 2.1 High-level component diagram

```text
                          ┌──────────────────────────────────────────────┐
   your application ────► │                  Brain (facade)               │
   (from_files / ask)     │  brain/brain.py — orchestrates everything     │
                          └───┬───────────┬───────────┬───────────┬───────┘
                              │           │           │           │
              ┌───────────────▼──┐  ┌─────▼─────┐ ┌───▼────────┐ ┌▼───────────────┐
              │  Storage          │  │ Processor │ │  Embedder  │ │  Vector store   │
              │  StorageBase      │  │ registry  │ │ Embeddings │ │  VectorStore    │
              │  ├ Transparent    │  │ (lazy)    │ │ (OpenAI)   │ │  (FAISS default)│
              │  └ LocalStorage   │  └─────┬─────┘ └────────────┘ └────────┬────────┘
              └───────────────────┘        │                               │
                                           ▼                               ▼
            INGESTION: file ─► parse ─► split (SplitterConfig) ─► embed ─► store
                                                                            │
                                                                            ▼
                          ┌──────────────────────────────────────────────────────┐
   RETRIEVAL per question │   QuivrQARAGLangGraph  (rag/quivr_rag_langgraph.py)    │
                          │   LangGraph StateGraph compiled from WorkflowConfig    │
                          │   nodes: filter_history → rewrite → retrieve →         │
                          │          [tool_routing → run_tool] → generate_rag      │
                          │   LLMEndpoint (multi-supplier) + RerankerConfig +      │
                          │   prompts registry + LLMToolFactory (web search…)      │
                          └───────────────────────────────┬──────────────────────┘
                                                           ▼
                                          ParsedRAGResponse (answer + citations + sources)
                          Observability: Langfuse callback handler (global)
```

### 2.2 Two execution phases

Quivr has a clean split between a one-time **ingestion** phase and a per-question **retrieval** phase.

**Ingestion** (`Brain.afrom_files` → `process_files`):
1. Each path is wrapped in a `QuivrFile` (`load_qfile`) — computes a SHA-1, detects extension via mimetypes.
2. The file is placed in `storage` (memory or disk).
3. `get_processor_class(extension)` resolves a processor (lazy import, priority-ordered).
4. The processor reads the file, splits it (`RecursiveCharacterTextSplitter`, token-based), and emits LangChain `Document` chunks; each chunk is enriched with metadata (chunk index, language, file SHA-1, processor class).
5. Chunks are embedded and added to the vector store (FAISS built via `FAISS.afrom_documents`).

**Retrieval** (`Brain.ask_streaming` → `QuivrQARAGLangGraph.answer_astream`):
1. The configured `WorkflowConfig` is compiled into a LangGraph `StateGraph`.
2. The graph runs node by node over an `AgentState` (messages, chat history, tasks, files…).
3. The final (`generate_rag`) node's tokens are streamed back; the last chunk carries metadata.

### 2.3 The workflow-as-graph design (the key idea)

The most important architectural decision is that **retrieval logic is data, not code**. A `WorkflowConfig` is a list of `NodeConfig`s, each naming a method on `QuivrQARAGLangGraph` and declaring its outgoing `edges` or a `conditional_edge`. At runtime `_build_workflow` calls `getattr(self, node.name)` to bind each node to its implementing method and wires the edges into a LangGraph `StateGraph`.

Consequences:
- Adding web search, dynamic retrieval, or prompt-editing is a YAML change, not a code change — as long as the node names map to existing methods.
- The set of *available* node behaviours is fixed by the methods on `QuivrQARAGLangGraph` (see §6.3). You cannot invent arbitrary node names; an unknown name raises `AttributeError` at graph build.
- The first node must be `START`; the validator `check_first_node_is_start` enforces this.

### 2.4 Two RAG engines

| Engine | File | Status | Mechanism |
|--------|------|--------|-----------|
| `QuivrQARAGLangGraph` | `rag/quivr_rag_langgraph.py` | **Default / primary** | LangGraph state machine compiled from `WorkflowConfig`. |
| `QuivrQARAG` | `rag/quivr_rag.py` | Legacy | A fixed LangChain Expression Language (LCEL) chain. |

> [!NOTE]
> `Brain.ask_streaming` accepts a `rag_pipeline` argument, but the implementation always instantiates `QuivrQARAGLangGraph`. The argument is effectively vestigial — the LangGraph engine is what runs. Treat `QuivrQARAG` as deprecated for new work.

---

## 3. Requirements

### 3.1 Runtime

| Requirement | Value |
|-------------|-------|
| Python | **≥ 3.11** (`requires-python = ">= 3.11"`) |
| Default LLM | OpenAI `gpt-4o` (needs `OPENAI_API_KEY`) |
| Default embedder | `OpenAIEmbeddings` |
| Default vector store | `faiss-cpu` (in-memory) |
| License | Apache 2.0 |

### 3.2 Core dependencies (`core/pyproject.toml`)

| Dependency | Constraint | Role |
|------------|-----------|------|
| `pydantic` | `>=2.8.2` | Config models, serialization. |
| `langchain-core` | `>=0.3,<0.4` | Documents, messages, prompts, runnables. |
| `langchain` | `>=0.3.9,<0.4` | Retrievers, compressors. |
| `langgraph` | `>=0.2.38,<0.3` | The workflow state machine. |
| `langchain-openai` | `>=0.3.0` | Default LLM + embeddings. |
| `langchain-anthropic` / `-mistralai` / `-google-genai` / `-groq` / `-cohere` / `-community` | various | Provider clients + Tavily/Jina tools. |
| `faiss-cpu` | `>=1.8.0.post1` | Default vector store. |
| `tiktoken` | `>=0.7.0` | Token counting. |
| `transformers[sentencepiece]` | `>=4.44.2` | Provider-specific tokenizers. |
| `megaparse-sdk` | `>=0.1.11` | MegaParse document parsing client. |
| `fasttext-langdetect` | `>=1.0.5` | Per-chunk language detection. |
| `langfuse` | `>=2.57.0` | Observability/tracing. |
| `rapidfuzz` | `>=3.10.1` | Fuzzy matching for tool-name validation. |
| `httpx`, `aiofiles`, `rich` | — | HTTP (Tika), async file IO, console output. |

### 3.3 Optional external services

| Service | Needed for | Configure with |
|---------|-----------|----------------|
| Apache Tika server | Default PDF parsing | `TIKA_SERVER_URL` (default `http://localhost:9998/tika`) |
| MegaParse (NATS) | Alternative document parsing | `MegaparseConfig` |
| Cohere / Jina | Re-ranking | `COHERE_API_KEY` / `JINA_API_KEY` |
| Tavily | Web search tool | `TAVILY_API_KEY` |
| Langfuse | Tracing | Langfuse env vars (host/keys) |

---

## 4. Configuration Mastery

All configuration is expressed as **Pydantic v2 models** that subclass `QuivrBaseConfig`. Every config object can be loaded from YAML via `Config.from_yaml(path)`.

> [!IMPORTANT]
> `QuivrBaseConfig` sets `model_config = ConfigDict(extra="forbid")`. **Unknown keys raise a validation error.** This is why a YAML containing `max_input_tokens` (a key that does not exist) fails — even though the project's own README shows it. Always validate keys against the model fields below; the in-repo `core/example_workflows/*.yaml` files are the correct reference.

### 4.1 Configuration object hierarchy

```text
AssistantConfig
├── retrieval_config: RetrievalConfig
│   ├── llm_config: LLMEndpointConfig
│   ├── reranker_config: RerankerConfig
│   └── workflow_config: WorkflowConfig
│       └── nodes: list[NodeConfig]
│           └── conditional_edge: ConditionalEdgeConfig
└── ingestion_config: IngestionConfig
    └── parser_config: ParserConfig
        ├── splitter_config: SplitterConfig
        └── megaparse_config: MegaparseConfig
```

### 4.2 `RetrievalConfig`

| Field | Type | Default | Meaning |
|-------|------|---------|---------|
| `llm_config` | `LLMEndpointConfig` | `LLMEndpointConfig()` | The answering model. |
| `reranker_config` | `RerankerConfig` | `RerankerConfig()` | Re-ranking (off unless `supplier` set). |
| `workflow_config` | `WorkflowConfig` | default RAG graph | The retrieval graph. |
| `max_history` | `int` | `10` | Conversation pairs kept in context. |
| `max_files` | `int` | `20` | Max files listed in the prompt. |
| `k` | `int` | `40` | Chunks fetched by the retriever. |
| `prompt` | `str \| None` | `None` | Custom system instructions. |

### 4.3 `LLMEndpointConfig`

| Field | Type | Default | Notes |
|-------|------|---------|-------|
| `supplier` | `DefaultModelSuppliers` | `openai` | Auto-inferred from `model` if not set. |
| `model` | `str` | `"gpt-4o"` | Model name. |
| `temperature` | `float` | `0.3` | Sampling temperature. |
| `max_context_tokens` | `int` | `20000` | Prompt budget; auto-capped to model limit; min 4096. |
| `max_output_tokens` | `int` | `4096` | Answer length; auto-capped; min 4096. |
| `streaming` | `bool` | `True` | Token streaming. |
| `llm_base_url` | `str \| None` | `None` | Custom endpoint (Ollama/Azure/proxy). |
| `llm_api_key` | `str \| None` | `None` | Falls back to `env_variable_name`. |
| `env_variable_name` | `str \| None` | `<SUPPLIER>_API_KEY` | Auto-derived key name. |
| `tokenizer_hub` | `str \| None` | per-model | HF tokenizer for token counting. |

Behaviour worth knowing:
- **Model-aware capping** (`set_llm_model_config`): Quivr looks up the model in a built-in `LLMModelConfig._model_defaults` table (per supplier) and lowers `max_context_tokens`/`max_output_tokens` to the model's real limits, logging a warning. Values below `MIN_CONTEXT_TOKENS`/`MIN_OUTPUT_TOKENS` (both **4096**) raise `ValueError`.
- **Supplier inference** (`set_llm_model`): the supplier is derived by `startswith` matching against the model table (e.g. `claude-*` → Anthropic).
- **Instance caching**: `LLMEndpoint.from_config` caches instances by the config's `__hash__`, so identical configs reuse the same client.

Supported suppliers (`DefaultModelSuppliers`): `openai`, `azure`, `anthropic`, `meta`, `mistral`, `groq`, `gemini`. Each maps to a LangChain chat client in `LLMEndpoint.from_config`.

### 4.4 `RerankerConfig`

| Field | Type | Default | Meaning |
|-------|------|---------|---------|
| `supplier` | `DefaultRerankers \| None` | `None` | `cohere` or `jina`; `None` = no re-ranking. |
| `model` | `str \| None` | supplier default | Cohere→`rerank-v3.5`, Jina→`jina-reranker-v2-base-multilingual`. |
| `top_n` | `int` | `5` | Chunks kept after re-ranking. |
| `relevance_score_threshold` | `float \| None` | `None` | Drop chunks below this score (0–1). |
| `relevance_score_key` | `str` | `"relevance_score"` | Metadata key holding the score. |

> [!WARNING]
> When `supplier` is set, `RerankerConfig` validates that `<SUPPLIER>_API_KEY` exists in the environment **at construction time** and raises `ValueError` if missing. A workflow that names a reranker supplier will fail fast without the key.

### 4.5 `SplitterConfig` and `MegaparseConfig`

| `SplitterConfig` | Default | | `MegaparseConfig` | Default |
|------------------|---------|--|-------------------|---------|
| `chunk_size` | `400` (tokens) | | `method` | `unstructured` |
| `chunk_overlap` | `100` (tokens) | | `strategy` | `fast` (`auto`/`hi_res`) |
| | | | `check_table` | `False` |
| | | | `model_name` | `gpt-4o` |

### 4.6 `WorkflowConfig`, `NodeConfig`, `ConditionalEdgeConfig`

- `WorkflowConfig`: `name`, `nodes: list[NodeConfig]`, `available_tools: list[str]` (validated against the tool registry — an unknown tool raises a "did you mean…" error via fuzzy matching), plus internal `validated_tools` / `activated_tools`.
- `NodeConfig`: `name` (must equal a `QuivrQARAGLangGraph` method, or `START`/`END`), `edges` or `conditional_edge`, optional `tools` (instantiated via `LLMToolFactory`), `description`.
- `ConditionalEdgeConfig`: `routing_function` (a method name like `routing_split` or `tool_routing`) and `conditions` (a list/dict of next-node names, with `START`/`END` resolved to LangGraph sentinels).

### 4.7 Environment variable reference

| Variable | Used by | Default |
|----------|---------|---------|
| `OPENAI_API_KEY` | OpenAI LLM/embeddings | — |
| `ANTHROPIC_API_KEY` | Anthropic | — |
| `MISTRAL_API_KEY` | Mistral | — |
| `GEMINI_API_KEY` | Gemini | — |
| `GROQ_API_KEY` | Groq | — |
| `AZURE_API_KEY` | Azure OpenAI | — |
| `COHERE_API_KEY` / `JINA_API_KEY` | Re-rankers | — |
| `TAVILY_API_KEY` | Web search tool | — |
| `TIKA_SERVER_URL` | PDF parsing | `http://localhost:9998/tika` |
| `QUIVR_LOCAL_STORAGE` | `LocalStorage` directory | `~/.cache/quivr/files` |
| `TOKENIZERS_PARALLELISM` | HF tokenizers | set to `false` to silence fork warnings |

> [!NOTE]
> There is **no central env-var scrape and no `.env` loaded by the library itself.** API keys are read on demand via `os.getenv` inside the config models (`set_api_key`). Your application is responsible for populating the environment (the examples use `python-dotenv`). Missing keys are logged as warnings (the key *value* is never logged).

---

## 5. Repository Structure

```text
quivr/
├── core/                         # the quivr-core package
│   ├── quivr_core/
│   │   ├── brain/                # Brain facade + info + (de)serialization + defaults
│   │   ├── llm/                  # LLMEndpoint multi-supplier client + tokenizer cache
│   │   ├── llm_tools/            # tool registry (web search / Tavily, cited_answer)
│   │   ├── processor/            # parser registry + implementations + splitter
│   │   ├── rag/                  # RAG engines, prompts, entities (config/models/chat)
│   │   ├── storage/              # StorageBase + LocalStorage/TransparentStorage
│   │   ├── files/                # QuivrFile abstraction
│   │   ├── language/             # language enum + fasttext detection
│   │   ├── base_config.py        # QuivrBaseConfig (from_yaml, extra=forbid)
│   │   └── config.py             # MegaparseConfig, ParserType, StrategyEnum
│   ├── tests/                    # pytest suite (markers: base/tika/unstructured/slow)
│   ├── example_workflows/        # canonical workflow YAML (correct keys)
│   └── pyproject.toml            # package metadata + deps
├── docs/                         # mkdocs-material documentation site
└── examples/                     # chatbot, chatbot_voice, quivr-whisper, simple_question
```

> [!TIP]
> The codebase is organized by concern, and each concern is fronted by an abstract base class (`StorageBase`, `ProcessorBase`) or a registry (`processor/registry.py`, `llm_tools/llm_tools.py`). To extend Quivr you implement an ABC or register into a registry — you rarely modify existing files.

---

## 6. Module Reference

### 6.1 `brain/brain.py` — the `Brain` facade

The orchestrator. Key surface:

| Member | Kind | Purpose |
|--------|------|---------|
| `from_files` / `afrom_files` | classmethod | Ingest files → build brain. |
| `afrom_langchain_documents` | classmethod | Build from pre-made `Document`s. |
| `load` / `save` | classmethod / async | Persist & restore a brain (FAISS + `config.json`). |
| `ask` / `aask` | method | One-shot answer (`ParsedRAGResponse`). |
| `ask_streaming` | async generator | Streamed answer (`ParsedRAGChunkResponse`). |
| `asearch` | async | Similarity search → `list[SearchResult]`. |
| `info` / `print_info` | method | Rich tree of brain metadata. |
| `chat_history` / `get_chat_history` | property/method | Access conversation state. |

`process_files(storage, skip_file_error, **kwargs)` is the module-level ingestion helper: it iterates storage, resolves a processor per file, and concatenates the resulting chunks.

`brain_defaults.py` supplies `default_llm()` (OpenAI gpt-4o), `default_embedder()` (`OpenAIEmbeddings`), and `build_default_vectordb()` (FAISS), each raising an actionable `ImportError` if the optional dependency is missing.

### 6.2 `llm/llm_endpoint.py` — provider abstraction

- `LLMEndpoint.from_config(config)` builds the right LangChain chat client for the supplier (`ChatOpenAI`, `AzureChatOpenAI`, `ChatAnthropic`, `ChatMistralAI`, `ChatGoogleGenerativeAI`, `ChatGroq`) and caches instances by config hash.
- `supports_func_calling()` returns `False` for `llama2`, `ollama3`, and `test` — driving the non-tool answering path.
- `count_tokens()` uses a per-model HF tokenizer with an LRU-style cache (`LLMTokenizer`, capped at 50 MB / 5 tokenizers) and a `cl100k_base` tiktoken fallback.
- `info()` returns an `LLMInfo` dataclass (used by `print_info`).

### 6.3 `rag/quivr_rag_langgraph.py` — the workflow engine

`QuivrQARAGLangGraph` implements every node behaviour as a method. The available nodes/routers:

| Method (node) | Role |
|---------------|------|
| `filter_history` | Trim chat history to the token budget. |
| `rewrite` | Condense each task into a standalone question (async, per-task). |
| `retrieve` | Fetch chunks via the (optionally re-ranked) compression retriever. |
| `dynamic_retrieve` | Iteratively widen `top_n`/`k` until enough relevant chunks (max 3 iterations). |
| `tool_routing` (router) | Decide per task whether a tool is needed. |
| `run_tool` | Execute the selected tool (e.g. Tavily web search). |
| `edit_system_prompt` | Update the system prompt + activate/deactivate tools from user instructions. |
| `generate_rag` | Build the RAG prompt, reduce context to fit, generate the answer. |
| `generate_chat_llm` | Answer without retrieval (plain chat). |
| `generate_zendesk_rag` | Specialized Zendesk-ticket answering node. |
| `retrieve_full_documents_context` | Enterprise-only: fetch full docs by knowledge id. |
| `routing` / `routing_split` (routers) | Split user input into instructions vs. tasks. |

Supporting structures: `AgentState` (the LangGraph state `TypedDict`), `UserTasks`/`UserTaskEntity` (multi-task tracking), `reduce_rag_context` (shrinks context to 85% of the limit by dropping history then least-relevant docs), and `(a)invoke_structured_output` (Pydantic-typed LLM calls with a JSON-schema-then-fallback strategy).

> [!NOTE]
> `retrieve_full_documents_context` asserts the vector store exposes `get_vectors_by_knowledge_id` and is documented in-code as "an enterprise only feature." With the default FAISS store this node will fail — only enable it with a compatible store.

### 6.4 `rag/prompts.py` — prompt registry

A frozen `MappingProxyType` of named `ChatPromptTemplate`s built once at import: `RAG_ANSWER_PROMPT`, `CONDENSE_TASK_PROMPT`, `SPLIT_PROMPT`, `TOOL_ROUTING_PROMPT`, `USER_INTENT_PROMPT`, `UPDATE_PROMPT`, `CHAT_LLM_PROMPT`, `DEFAULT_DOCUMENT_PROMPT`, and the `ZENDESK_*` prompts. The default RAG prompt hard-codes the grounding policy ("use ONLY the provided context… do not apologize… don't cite the source id in the answer").

### 6.5 `processor/` — document parsing

- `registry.py`: maps extensions → processor class paths with **lazy import** and a **min-heap priority** so optional/heavy parsers (MegaParse highest priority) are tried first and fall back to lighter ones on `ImportError`. `get_processor_class`, `register_processor`, `available_processors` are the public API.
- `implementations/default.py`: `_build_processor` factory dynamically generates processor classes wrapping LangChain document loaders (`CSVLoader`, `Docx2txtLoader`, `UnstructuredPDFLoader`, `PythonLoader`, `NotebookLoader`, etc.).
- `implementations/simple_txt_processor.py`: dependency-free `.txt` processor (works with the base install).
- `implementations/tika_processor.py`: PDF via a Tika HTTP server (3 retries, configurable URL/timeout).
- `implementations/megaparse_processor.py`: MegaParse over NATS for many formats.
- `processor_base.py`: `ProcessorBase` ABC; `process_file` enriches chunk metadata (chunk index, `quivr_core` version, detected language, file metadata) and sanitizes null bytes/encoding.
- `splitter.py`: `SplitterConfig` (chunk size/overlap).

### 6.6 `storage/`, `files/`, `language/`

- `storage_base.py` — `StorageBase` ABC (`nb_files`, `get_files`, `upload_file`, `remove_file`, `info`).
- `local_storage.py` — `LocalStorage` (copies or symlinks to a managed dir, de-dupes by SHA-1) and `TransparentStorage` (in-memory; the default).
- `files/file.py` — `QuivrFile` (slots-based) + `QuivrFileSerialized` (Pydantic) + `FileExtension` enum (~24 extensions) + `load_qfile`.
- `language/` — a ~170-entry `Language` enum and `detect_language` via fastText.

### 6.7 `rag/entities/`

- `config.py` — all the config models (§4).
- `models.py` — response/IO models: `cited_answer` (the citation tool schema), `ParsedRAGResponse`, `ParsedRAGChunkResponse`, `RAGResponseMetadata`, `QuivrKnowledge`, `SearchResult`, `ChatMessage`, `LangchainMetadata`.
- `chat.py` — `ChatHistory` (append, `iter_pairs`, `to_list`, time-ordered retrieval).

---

## 7. Security Analysis

> [!CAUTION]
> **Pickle deserialization on `Brain.load`.** Restoring a saved brain calls `FAISS.load_local(..., allow_dangerous_deserialization=True)`, which unpickles the index. A maliciously crafted saved brain can execute arbitrary code on load. **Only load brains your application produced or fully trusts.** Never expose an endpoint that loads a user-supplied brain directory.

### 7.1 Threat model

Quivr is a library running inside your trusted process. It does **not** provide authentication, authorization, multi-tenant isolation, or request rate limiting — those are the hosting application's responsibility (the LlamaIndex/LangChain trust model).

| Concern | Quivr's behaviour | Your responsibility |
|---------|-------------------|---------------------|
| Secrets | Read from env via `os.getenv`; wrapped in `SecretStr` for clients; missing keys logged as warnings (values never logged). `Brain.save` excludes the OpenAI API key from embedder serialization. | Inject keys securely; don't commit `.env`; rotate keys. |
| Untrusted documents | Parsed by third-party loaders (`unstructured`, MegaParse, Tika). | Sandbox/limit parsing of untrusted uploads; the parsers are large attack surfaces. |
| Prompt injection | Default prompt restricts the model to provided context, but retrieved documents can still contain adversarial instructions. | Treat ingested content as untrusted; consider output filtering. |
| Data egress | Document chunks and metadata are sent to your configured LLM/embedding/reranker/web-search providers by default. | Choose providers per your data-residency needs; use local models for sensitive data. |
| Network calls | Tika (`TIKA_SERVER_URL`), Tavily, Langfuse, and provider APIs. | Control egress; pin/validate the Tika endpoint. |
| Saved-brain pickle | `allow_dangerous_deserialization=True` (above). | Only load trusted brains. |

### 7.2 Observability and PII

A global Langfuse `CallbackHandler` is attached to LLM calls (`LangfuseService`). If Langfuse env vars are configured, prompts, contexts, and answers are sent to Langfuse for tracing. Disable or self-host Langfuse if your data cannot leave your perimeter.

---

## 8. Operations

### 8.1 Embedding Quivr in a service

Quivr has no server of its own — you run it inside yours (FastAPI, Flask, a worker, etc.). Operational guidance:

- [ ] Run all Quivr operations on the **async** path inside async web frameworks (`afrom_files`, `aask`, `ask_streaming`) to avoid the event-loop error (§9).
- [ ] Build brains **once** and persist them with `save()`; reload with `load()` to avoid repeated embedding cost and latency.
- [ ] Cache `Brain` instances per user/session (the `quivr-whisper` example keeps a `dict` of brains keyed by session id).
- [ ] Pre-warm tokenizers with `LLMTokenizer.preload_tokenizers([...])` at startup if you count tokens on a hot path.
- [ ] Pin `quivr-core` and the `langchain*`/`langgraph` versions — the LangChain ecosystem moves fast and the constraints are narrow (`<0.4`).

### 8.2 Persistence layout

`brain.save(folder)` writes `folder/brain_<id>/` containing `config.json` (a `BrainSerialized`: id, name, chat history, LLM config, vector-store config, embedder config, storage config) and a `vector_store/` FAISS index. `Brain.load(folder/brain_<id>)` reconstructs from those.

> [!NOTE]
> `save()`/`load()` currently support only the **FAISS** vector store, the **OpenAI** embedder, and `TransparentStorage`/`LocalStorage`. Other combinations raise `"can't serialize ... for now"`. For Postgres/pgvector and other stores you must manage persistence yourself.

### 8.3 Testing the library

```bash
# Inside core/, with dev deps installed (rye sync) and a base env:
pytest -m base               # tests needing the `base` extra
pytest -m "not slow"         # skip slow tests
```

Markers (`core/pyproject.toml`): `base`, `tika` (needs a Tika server), `unstructured` (needs the extra), `slow`. A Dockerized run is provided via `core/scripts/run_tests.sh` (builds `Dockerfile.test`, runs `tox`).

### 8.4 Building the docs site

```bash
cd docs
rye sync
rye run docs          # mkdocs serve
rye run build_docs    # mkdocs build --strict
```

---

## 9. Troubleshooting (Operational)

### Symptom: `RuntimeError: There is no current event loop`
**Cause:** Synchronous wrapper (`from_files`/`ask`) invoked where an event loop already runs (ASGI handler, notebook). The wrappers use `asyncio.get_event_loop().run_until_complete`. (Issue [#3650](https://github.com/QuivrHQ/quivr/issues/3650).)
**Fix:** Use `afrom_files`/`aask`/`ask_streaming`. **Prevent:** Standardize on async in service code.

### Symptom: asyncpg / `CancelledError` / connection-pool warnings
**Cause:** Async lifecycle issues when using a Postgres-backed vector store outside a properly scoped event loop (issues [#3654](https://github.com/QuivrHQ/quivr/issues/3654), [#3557](https://github.com/QuivrHQ/quivr/issues/3557)).
**Fix:** Ensure DB connections are created and closed within the same running loop; scope the pool to the request/worker lifecycle. **Prevent:** Manage the event loop explicitly in long-running services.

### Symptom: Pydantic `ValidationError` loading a workflow YAML
**Cause:** An unknown key (e.g. `max_input_tokens`) — models use `extra="forbid"`.
**Fix:** Use the documented field names (§4); start from `core/example_workflows/*.yaml`.

### Symptom: `ValueError: The API key for supplier '<x>' is not set`
**Cause:** Reranker/LLM supplier configured without its env key (rerankers validate at construction).
**Fix:** Export the named variable before building the config.

### Symptom: `RuntimeError: can't send parse request to tika server`
**Cause:** No reachable Tika server for PDFs (3 retries exhausted).
**Fix:** Start `apache/tika` or set `TIKA_SERVER_URL`; or install an alternative PDF parser.

### Symptom: `max_context_tokens is too low`
**Cause:** Token limit set below 4096.
**Fix:** Use ≥ 4096 for both context and output tokens.

---

## 10. Code Quality & Patterns

**Strengths**
- Clean separation of concerns behind ABCs (`StorageBase`, `ProcessorBase`) and registries — extensible without editing core files.
- Config-as-data: the entire retrieval pipeline is declarative YAML compiled to LangGraph, a genuinely flexible design.
- Async-first with thin sync wrappers; comprehensive Pydantic validation with helpful error messages.
- Provider abstraction isolates six LLM vendors behind one `LLMEndpoint`.
- A real pytest suite with environment markers (`base`/`tika`/`unstructured`).

**Weaknesses / tech debt (from in-code `TODO`/`FIXME` and observed behaviour)**
- The sync wrappers' `get_event_loop().run_until_complete` pattern is fragile in modern async runtimes (the #1 reported bug).
- `Brain.add_file()` is declared but `raise NotImplementedError` — files can only be added at construction.
- `LocalStorage.remove_file` / `TransparentStorage.remove_file` are `NotImplementedError`.
- `save()`/`load()` are hard-wired to FAISS + OpenAI embedder; other stores aren't serializable.
- `parse_chunk_response` is annotated in-code as "CONVOLUTED LOGIC / redo this."
- The legacy `QuivrQARAG` engine duplicates retrieval logic now owned by the LangGraph engine.
- The project's own README/docs YAML examples use the invalid `max_input_tokens` key (contradicting the code and the in-repo example workflows).
- `quivr_rag.py` notes a lingering dependency on the umbrella `langchain` package the maintainers want to drop.

---

## 11. Suggested Improvements

1. **Replace the sync event-loop wrappers** with `anyio`/`asyncio.run`-safe execution (or document async-only), closing the most common failure mode.
2. **Implement `Brain.add_file` and `remove_file`** so brains are mutable without a full rebuild.
3. **Generalize serialization** so `save()`/`load()` support any vector store and embedder via a registry, mirroring the processor design.
4. **Fix the documentation YAML keys** (`max_input_tokens` → `max_output_tokens`) and add a config-schema doc generated from the Pydantic models.
5. **Retire `QuivrQARAG`** or clearly mark it deprecated and remove the vestigial `rag_pipeline` argument from `ask_streaming`.
6. **Harden untrusted-document ingestion** with sandboxing guidance and optional parser timeouts/limits.
7. **Make the saved-brain format pickle-free** (or signed) to remove the deserialization risk.

---

## 12. Contributing

- Repository: `github.com/QuivrHQ/quivr`; the package lives in `core/`.
- Tooling (dev deps in `core/pyproject.toml`): `ruff`, `flake8` + `flake8-black`, `mypy`, `pre-commit`, `pytest` (+ `pytest-asyncio`, `pytest-xdist`, `pytest-benchmark`).
- Build backend: `hatchling`; environments managed with `rye` (lockfiles checked in).
- Run the suite via `pytest` or the Dockerized `core/scripts/run_tests.sh`.
- License: Apache 2.0.

> [!NOTE]
> Public-issue activity has slowed and the community has asked whether the project is still maintained (issue [#3681](https://github.com/QuivrHQ/quivr/issues/3681)). For production use, pin a specific `quivr-core` version and vendor/patch as needed.

---

## 13. Quick Reference

### Default values at a glance

| Setting | Default |
|---------|---------|
| LLM model / supplier | `gpt-4o` / `openai` |
| Temperature | `0.3` |
| `max_context_tokens` / `max_output_tokens` | `20000` / `4096` (min 4096) |
| Embedder | `OpenAIEmbeddings` |
| Vector store | in-memory FAISS |
| Storage | `TransparentStorage` (in-memory) |
| Chunk size / overlap | `400` / `100` tokens |
| Retriever `k` | `40` |
| Reranker `top_n` | `5` (off unless supplier set) |
| `max_history` / `max_files` | `10` / `20` |

### Essential APIs

```python
from quivr_core import Brain
from quivr_core.llm import LLMEndpoint
from quivr_core.rag.entities.config import (
    RetrievalConfig, LLMEndpointConfig, IngestionConfig,
)

brain = await Brain.afrom_files(name="b", file_paths=[...])   # build
cfg = RetrievalConfig.from_yaml("workflow.yaml")              # configure
async for c in brain.ask_streaming("q", run_id=uuid4(),       # answer
                                   retrieval_config=cfg):
    ...
await brain.save("path"); Brain.load("path/brain_<id>")       # persist
```

---

_Generated June 2026 from QuivrHQ/quivr at 947a785. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._