TextGen (text-generation-webui) User Guide

TextGen is a self-hosted web interface for running large language models on your own computer. You point it at a model file, pick how you want to talk to it — a chat window, a freeform notebook, or an OpenAI-compatible API — and it handles loading the model, managing context, sampling, and streaming the reply. It runs locally with no account, no cloud dependency, and no telemetry.

A single install can load models in five different runtimes (llama.cpp, Transformers, ExLlamaV3, and TensorRT-LLM), generate images, transcribe audio, search the web mid-chat, call tools over MCP, and serve the whole thing to other applications through a drop-in replacement for the OpenAI and Anthropic APIs.

Note

This guide covers what TextGen does and how to use it day to day. For architecture, the full configuration reference, deployment, and security hardening, see the companion Admin Guide.

1. Quick Start

The fastest path is a portable build — a self-contained zip that needs no installation and runs models through llama.cpp. It opens as a desktop app.

Windows

1. Download the latest textgen-portable-*-windows-*.zip from the project's releases.
2. Extract it anywhere without spaces in the path.
3. Double-click start_windows.bat (or the bundled desktop launcher).

Linux

unzip textgen-portable-*-linux-*.zip
cd textgen-portable-*
./start_linux.sh

macOS

unzip textgen-portable-*-macos-*.zip
cd textgen-portable-*
./start_macos.sh

On first launch TextGen tells you that no model is downloaded yet:

You haven't downloaded any model yet.
Once the web UI launches, head over to the "Model" tab and download one.

Open the Model tab, paste a Hugging Face model ID (for example bartowski/Llama-3.2-3B-Instruct-GGUF) into the download box, download it, select it from the dropdown, and click Load. Switch to the Chat tab and start typing.

Tip

Portable builds run GGUF models through llama.cpp only. If you want Transformers, ExLlamaV3, image generation, or LoRA training, use the one-click installer instead (see Installation).

2. Requirements

Important

A GPU is strongly recommended but not required. TextGen can offload some or all model layers to the GPU and keep the rest in CPU RAM, so you can run models larger than your VRAM — just more slowly.

3. Installation

TextGen offers three installation paths. Pick the one that matches how much you need.

3.1 Portable build (no install)

Already covered in Quick Start. Best for: trying TextGen quickly, running GGUF models, machines where you don't want a multi-gigabyte Python environment.

Capability trade-off: portable builds run with --portable, which hides features that aren't available in that mode:

3.2 One-click installer (full features)

The one-click installer downloads a private Python 3.13 environment via Miniforge/conda, installs PyTorch for your hardware, and sets everything up. It supports every loader and feature.

1. Download or git clone the repository.
2. Run the start script for your OS:

Windows

Double-click start_windows.bat.

Linux

./start_linux.sh

macOS

./start_macos.sh

WSL

Double-click start_wsl.bat.

3. On first run you'll be asked which hardware you have:

What is your GPU?

A) NVIDIA
B) AMD - Linux only, ROCm 7.2
C) Apple M Series
D) Intel Arc (beta)
N) CPU mode

Pick the letter that matches your machine. The installer remembers your choice in .installer_state.json and installs the matching requirements (requirements/full/requirements_*.txt). When it finishes, the web UI launches automatically.

Tip

To pre-select your GPU non-interactively (useful for scripted installs), set the GPU_CHOICE environment variable to A, B, C, D, or N before running the start script.

Caution

The conda-based installer relies on Miniforge, which cannot be installed under a path containing spaces. Install TextGen into a path like C:\textgen, not C:\My Programs\textgen.

3.3 Updating

Run the update wizard for your OS (update_wizard_windows.bat, update_wizard_linux.sh, update_wizard_macos.sh, or update_wizard_wsl.bat). It opens a menu:

What would you like to do?

A) Update the web UI
B) Install/update extensions requirements
C) Revert local changes to repository files with "git reset --hard"
N) Nothing (exit)

Choose A to pull the latest release and refresh dependencies.

Warning

Major Python or CUDA version bumps require a clean reinstall. If you see a message instructing you to delete the installer_files folder, do so and re-run the start script — the installer cannot upgrade across those boundaries in place.

4. Downloading Models

You have two ways to get models: the Model tab in the UI, or the download-model.py command-line tool. Both save into user_data/models/ (LoRAs go to user_data/loras/).

4.1 From the Model tab

Open the Model tab, paste a Hugging Face model ID into the download field, and download. TextGen automatically picks sensible files:

• If both PyTorch and safetensors weights exist, it downloads safetensors only.
• For GGUF repositories with many quantizations, it downloads the Q4_K_M file by default (a good size/quality balance) unless you specify a particular file.

4.2 From the command line

python download-model.py facebook/opt-1.3b

Downloading the model to user_data/models/facebook_opt-1.3b
Download of 5 files to user_data/models/facebook_opt-1.3b completed.

Download a specific GGUF quantization (recommended for large GGUF repos):

python download-model.py bartowski/Llama-3.2-3B-Instruct-GGUF --specific-file Llama-3.2-3B-Instruct-Q4_K_M.gguf

Common download-model.py options:

Tip

For gated or private Hugging Face models, set HF_TOKEN (or HF_USER / HF_PASS) in your environment before downloading. You can also point HF_ENDPOINT at a mirror.

Note

Downloads resume automatically if interrupted — re-run the same command. Use --check afterward to confirm every file's checksum matches Hugging Face, and --clean to force a fresh download if a checksum fails.

5. Loading a Model

Open the Model tab, choose your model from the dropdown, and click Load. TextGen auto-detects the right loader from the files present, but you can override it.

5.1 The five loaders

Tip

If you're unsure, just download a GGUF model and let the default llama.cpp loader handle it. It works on CPU, NVIDIA, AMD, Apple, and Intel hardware.

5.2 Fitting a model to your hardware

The most important load-time settings (all on the Model tab, and all available as CLI flags):

Warning

If loading fails with an out-of-memory error, reduce --gpu-layers, lower --ctx-size, or switch the cache type to q8_0. See Troubleshooting.

5.3 Idle unloading

Pass --idle-timeout 30 to automatically unload the model after 30 minutes of inactivity (freeing VRAM); it reloads automatically on your next request.

6. Chatting With a Model

The Chat tab is the main way to talk to a model. Type in the input box at the bottom and press Enter to send.

6.1 Chat modes

TextGen has three conversation modes (set in the chat controls or via the mode setting):

6.2 Everyday chat actions

Tip

Pressing → (right arrow) when you're already on the newest version of a reply regenerates a fresh one — a quick way to keep re-rolling.

6.3 Managing conversations

• New chat — starts a fresh conversation. The dropdown arrow next to it offers an Incognito chat that isn't saved to disk.
• Past chats — the left sidebar lists saved conversations. Hover a chat to reveal a trash icon for quick deletion.
• Chats are saved automatically (except in incognito or --multi-user mode).

6.4 Attachments and pasting long text

Use the upload button in the chat input to attach text files, PDFs, DOCX documents, and images (vision models can see images). If you paste a very long block of text, TextGen can automatically turn it into an attachment instead of cluttering the input — enable Paste to attachment in settings.

6.5 Web search

Click the web search toggle (a globe icon, tooltip "Search the internet with DuckDuckGo") to let the model search the web for your query. TextGen fetches the top results, extracts their main text, and adds them as attachments the model can read.

Using search query: latest mars rover mission
Added 3 successful web search results as attachments.

Control how many pages are fetched with the web_search_pages setting (default 3).

Note

Web search has built-in protection against being tricked into fetching internal/private network addresses, so it's safe to leave enabled.

7. The Notebook and Default Tabs

Not every task is a conversation. The Notebook tab gives you a single freeform text area where the model continues whatever you've written — ideal for story writing, code completion, or prompt experiments. The Default tab splits input and output into two boxes for classic prompt-in/completion-out workflows.

• Prompts are saved as .txt files under user_data/logs/notebook/ and can be reloaded from the prompt dropdown.
• Click Token count to see how many tokens your prompt uses against the model's context window.

Tip

Use the Notebook tab to test how a model behaves without a chat template — it's the closest thing to talking to the raw model.

8. Controlling the Output: Presets and Samplers

How creative, focused, or repetitive a model is comes down to sampler settings, grouped into reusable presets on the Parameters tab.

8.1 Using presets

Select a preset from the dropdown on the Parameters tab. TextGen ships several; Top-P is the default starting point. Changing a preset instantly applies its sampler values. You can also:

• Save the current sliders as a new preset (written to user_data/presets/<name>.yaml).
• Neutralize samplers to reset everything to neutral/off values.
• Reset back to the preset's saved values.

8.2 The most useful samplers

Tip

For factual/coding tasks, lower the temperature (e.g. 0.2–0.5). For creative writing, raise it (0.9–1.2) and enable a repetition control like dry_multiplier.

Note

Different loaders support different samplers. The Parameters tab automatically hides samplers your current loader doesn't implement, so you only see what will actually take effect. The Admin Guide has the full per-loader sampler matrix.

8.3 Grammars (constrained output)

In the Parameters tab you can supply a GBNF grammar to force the model's output into a specific shape — valid JSON, a restricted vocabulary, a list format, and so on. Several ready-made grammars ship in user_data/grammars/ (e.g. json.gbnf, list.gbnf).

9. Characters and Instruction Templates

The Character tab (under Parameters) lets you define who the model is.

• Character — a name, context/personality, and greeting. Switch personas from the dropdown; cards live in user_data/characters/.
• Your profile — your display name (name1) and an optional bio the model is told about you.
• Instruction template — the Jinja2 template that formats messages for the model. TextGen auto-selects the right one for most known models; templates live in user_data/instruction-templates/.

Important

If a model gives garbled or "off" replies, the wrong instruction template is the usual cause. Most models include their own template metadata that TextGen applies automatically, but you can override it on the Model or Character tab.

9.1 Reasoning / "thinking" models

For models that emit a hidden reasoning block (DeepSeek-R1, Qwen3, GPT-OSS, Gemma, and similar), TextGen automatically detects the thinking section and shows it collapsed, with the final answer below. Three settings control this:

10. Generating Images

Note

Image generation is available in full installs only (not portable builds).

The Image generation tab runs diffusion image models locally. Download an image model into user_data/image_models/, select it, enter a prompt, and generate.

Key controls: prompt and negative prompt, width/height (or aspect ratio), steps, CFG scale, seed, and batch size/count. You can also start the server with an image model preselected:

python server.py --image-model FLUX.1-schnell --image-quant bnb-4bit

Tip

Enable LLM prompt variations to have your loaded language model rewrite/expand your image prompt before generating — handy for richer results from a short prompt.

You can also generate images from chat or over the API (/v1/images/generations).

11. Using the API

TextGen can serve an OpenAI-compatible (and Anthropic-compatible) API, so any tool that talks to ChatGPT or Claude can talk to your local model. Start it with --api:

python server.py --api --api-key my-secret-key

Starting TextGen
OpenAI-compatible API URL:
http://127.0.0.1:5000

Run headless (API only, no web UI) with --nowebui.

11.1 Chat completions (OpenAI format)

curl http://127.0.0.1:5000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer my-secret-key" \
  -d '{
    "messages": [{"role": "user", "content": "Write a haiku about local LLMs."}],
    "temperature": 0.7,
    "max_tokens": 200
  }'

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "choices": [{"message": {"role": "assistant", "content": "Weights on my own drive..."}}]
}

Set "stream": true to receive tokens as server-sent events as they're generated.

11.2 Pointing existing tools at TextGen

Most OpenAI client libraries just need the base URL and key changed:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:5000/v1", api_key="my-secret-key")
resp = client.chat.completions.create(
    model="local",  # the loaded model is used regardless of this value
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

11.3 Anthropic-compatible endpoint

TextGen also serves /v1/messages in Anthropic's format. Authenticate with the x-api-key header (instead of Authorization: Bearer):

curl http://127.0.0.1:5000/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: my-secret-key" \
  -d '{"messages": [{"role": "user", "content": "Hi"}], "max_tokens": 100}'

11.4 What the API exposes

Important

By default the API only accepts connections from your own machine (localhost). To let other devices reach it you must pass --listen (or --public-api). When you do, always set --api-key. See the Admin Guide's Security section before exposing the API.

Tip

The model load/unload endpoints (/v1/internal/model/*) require the admin key. Set a separate --admin-key so untrusted API clients can generate text but can't swap your model.

12. Troubleshooting

Issue numbers below reference the project's GitHub issue tracker for additional context.

Error: libllama-common.so.0: cannot open shared object file

Cause: The bundled llama.cpp binary can't find its shared library, usually after copying a portable build between machines or a partial extraction (#7592). Solution: Re-extract the portable zip fresh; don't move individual files out of it. On Linux, ensure the installer_files/portable folder wasn't relocated. Prevention: Keep the portable build's folder structure intact.

A GGUF model fails to load

Cause: Corrupted or partial download, an outdated GGUF format, or insufficient VRAM/RAM (#7530). Solution: Re-download with python download-model.py <repo> --specific-file <file>.gguf and verify with --check. If you see "You are using an outdated GGUF, please download a new one", get a freshly re-quantized version. Lower --gpu-layers if it's a memory issue. Prevention: Prefer well-maintained GGUF repositories and validate checksums.

AMD GPU model won't load

Cause: AMD support requires Linux + ROCm 7.2; AMD on Windows is not supported by the installer (#7546). Solution: Use Linux for AMD GPUs, or fall back to --cpu mode on Windows. Prevention: Choose option B (AMD) in the installer only on Linux.

Failed to load the extension "coqui_tts"

Cause: The extension's Python requirements aren't installed (#7580). Solution: Run the update wizard → B) Install/update extensions requirements, or pip install -r extensions/coqui_tts/requirements.txt inside the environment. Prevention: Install extension requirements before enabling an extension.

Out-of-memory when loading or generating

Cause: Model + context + KV cache exceeds your VRAM/RAM. Solution: Lower --gpu-layers, reduce --ctx-size, set --cache-type q8_0, or enable --cpu-moe for MoE models. Prevention: Start with a smaller quantization (Q4_K_M) and a modest context size.

API requests return 401 Unauthorized

Cause: Missing or wrong key. The server expects Authorization: Bearer <key> (or x-api-key for /v1/messages). Solution: Pass the exact value you set with --api-key. Remember model load/unload needs the --admin-key. Prevention: Keep your key handy; set --api-key and --admin-key explicitly.

API returns Invalid host header or refuses remote connections

Cause: The API is localhost-locked unless you opt in (#7545 for proxy setups). Solution: Start with --listen (LAN) or --public-api (Cloudflare tunnel) plus --api-key. Behind a reverse proxy, also set --subpath. Prevention: Decide up front whether the API should be local-only or networked.

The model "talks to itself" or produces garbled output

Cause: Wrong instruction template or chat mode for the model. Solution: On the Model/Character tab, select the correct instruction template; try instruct mode. Most models carry their own template — reloading often fixes it. Prevention: Use models that include template metadata, and don't override the template unless you know it's needed.

Generation won't stop / VRAM stays used after closing (desktop app)

Cause: In some cases the llama-server subprocess lingers after exit (#7574). Solution: Fully quit the app; if VRAM stays occupied, end any leftover llama-server process. Press Esc to stop a runaway generation. Prevention: Use the app's normal quit; avoid force-killing the window mid-generation.

13. Frequently Asked Questions

Q: Does TextGen send my chats anywhere? No. Everything runs locally. Gradio analytics are disabled and there's no built-in telemetry. Web search (opt-in) and the --share/--public-api tunnels are the only features that reach the internet.

Q: What model formats are supported? GGUF (llama.cpp), Hugging Face safetensors/PyTorch (Transformers), EXL3 (ExLlamaV3), and TensorRT-LLM engines. GGUF is the easiest to start with.

Q: Can I run a model bigger than my GPU's VRAM? Yes. Lower --gpu-layers so only some layers go to the GPU and the rest stay in CPU RAM, or use --cpu for CPU-only. It'll be slower but it works.

Q: How do I make my command-line flags stick across restarts? Put them in user_data/CMD_FLAGS.txt, one flag per line. They're applied on every launch (command-line flags still take precedence). Interface settings can likewise be saved in user_data/settings.yaml.

Q: Where are my models, characters, and chat logs stored? Everything lives under user_data/ — models/, loras/, characters/, presets/, grammars/, and logs/. Back up that folder to preserve your setup.

Q: How do I use TextGen from another program? Start it with --api and point any OpenAI-compatible client at http://127.0.0.1:5000/v1. See Using the API.

Q: Can multiple people use one instance? Yes, with --multi-user, but it's meant for small trusted teams only: chat history isn't saved, the Stop button stops everyone's generation, and grayed-out actions can be bypassed client-side. Don't expose it publicly.

Q: What's the difference between the portable build and the installer? Portable is a no-install zip limited to llama.cpp/GGUF (plus the desktop window and API). The one-click installer adds the Transformers/ExLlamaV3/TensorRT-LLM loaders, image generation, and LoRA training.

Q: How do I get speech, translation, or other extras? Enable the bundled extensions (Whisper STT, Coqui/Silero TTS, Google Translate, the SuperboogaV2 RAG memory, image generation via Stable Diffusion API, etc.) from the Session tab, after installing their requirements via the update wizard.

Q: Can the model use tools or call external services? Yes. TextGen supports tool calling and MCP (Model Context Protocol) servers configured in user_data/mcp.json. Select tools in the chat controls; enable confirmation prompts with the confirm_tool_calls setting.

Q: How do I update without losing my models and settings? Run the update wizard and choose A. Your user_data/ folder is never touched by updates.

Q: The web UI is slow over a remote/proxy connection — what can I do? Heavy proxying (e.g. Cloudflare in front of a remote host) can slow the UI (#7545). Prefer a direct --listen connection on a trusted network, or the desktop app locally.