LocalAI User Guide

LocalAI is the open-source, self-hosted AI engine. It runs large language models, vision models, speech, image, and video generation on your own hardware — and exposes them behind APIs that are drop-in compatible with OpenAI, Anthropic, ElevenLabs, and Ollama. No GPU is required, no data leaves your infrastructure, and you only install the parts you actually use.

This guide is for people who want to run LocalAI and use it: install it, load models, call the API, and use the built-in features (chat, embeddings, speech, images, agents, and more). If you operate LocalAI in production — deployment, scaling, security hardening, distributed clusters, and internals — see the companion Admin Guide.

Note

LocalAI is licensed MIT and created by Ettore Di Giacinto (@mudler). The binary you run is called local-ai. The default API port is 8080.

Table of Contents

1. Quick Start
2. What LocalAI Does
3. Installation
4. Loading Models
5. Everyday Usage: Calling the API
6. Working with the Web UI
7. Feature Walkthroughs
8. Configuring a Model (YAML)
9. The local-ai Command Line
10. Common Tasks (Recipes)
11. Troubleshooting
12. FAQ
13. Reference

1. Quick Start

The fastest way to a working LocalAI is a container. This starts the server, the web UI, and the model gallery in one command.

CPU only

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

NVIDIA GPU (CUDA 12)

docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

AMD GPU (ROCm)

docker run -ti --name local-ai -p 8080:8080 \
  --device=/dev/kfd --device=/dev/dri --group-add=video \
  localai/localai:latest-gpu-hipblas

Intel GPU (oneAPI)

docker run -ti --name local-ai -p 8080:8080 \
  --device=/dev/dri/card1 --device=/dev/dri/renderD128 \
  localai/localai:latest-gpu-intel

Once the container is up, open the web UI:

http://localhost:8080

Pull and run your first model from the gallery, then chat with it:

# Inside the container (or with a local binary)
local-ai run llama-3.2-1b-instruct:q4_k_m

Call it like the OpenAI API:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.2-1b-instruct:q4_k_m",
    "messages": [{"role": "user", "content": "Say hello in three languages."}]
  }'

Expected response shape:

{
  "object": "chat.completion",
  "model": "llama-3.2-1b-instruct:q4_k_m",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "Hello! Bonjour! ¡Hola!"},
      "finish_reason": "stop"
    }
  ]
}

Tip

Already ran LocalAI before? Don't docker run again — restart the existing container with docker start -i local-ai. A fresh docker run with the same --name will fail because the name is taken.

Important

LocalAI automatically detects your GPU and downloads the matching backend on demand the first time a model needs it. The core image is small; backends (llama.cpp, vLLM, diffusers, whisper.cpp, …) are pulled only when used. The first run of a new model type may take a while as the backend image downloads.

2. What LocalAI Does

LocalAI is a small core, not a bundle. The core is a Go server that speaks several popular API dialects. Each model is served by a backend — a self-contained engine (llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX, kokoro, parakeet…) packaged as its own image and pulled only when a model requires it.

Compatible API surfaces

LocalAI implements multiple vendor dialects so existing client libraries work unchanged:

• OpenAI — /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/audio/*, /v1/images/*, /v1/models, /v1/responses (Open Responses API), /v1/realtime.
• Anthropic — POST /v1/messages.
• Ollama — /api/chat, /api/generate, /api/embed, /api/tags, /api/show, /api/ps, /api/version.
• ElevenLabs — /v1/text-to-speech/:voice-id, /v1/sound-generation.
• Jina — /v1/rerank.

Tip

Point any OpenAI SDK at LocalAI by overriding the base URL to http://localhost:8080/v1 and using any API key (or a real one if you've enabled auth). The same trick works for Anthropic and Ollama SDKs against their respective paths.

3. Installation

3.1 Containers (recommended)

Containers ship everything needed and handle GPU plumbing. Pick the image tag that matches your hardware:

Persist your models and data with a volume so they survive container restarts:

docker run -ti --name local-ai -p 8080:8080 \
  -v $PWD/models:/models \
  -v $PWD/data:/data \
  localai/localai:latest

Note

docker run -ti keeps the container in the foreground so you can watch logs. Add -d to run detached. Use docker logs -f local-ai to follow logs afterward.

3.2 macOS desktop app

Download the DMG from the latest release. It bundles a desktop Launcher (a Fyne GUI with a system-tray icon) that starts/stops the server for you.

Warning

The DMG is not signed by Apple. After installing, clear the quarantine attribute or macOS will refuse to open it:

sudo xattr -d com.apple.quarantine /Applications/LocalAI.app

3.3 Verifying the install

After the server starts, confirm it's healthy:

curl http://localhost:8080/readyz

OK

Check the version:

curl http://localhost:8080/version

List loaded/known models:

curl http://localhost:8080/v1/models

Setup verification checklist:

• [ ] GET /readyz returns OK
• [ ] Web UI loads at http://localhost:8080
• [ ] GET /v1/models returns a (possibly empty) model list
• [ ] A model from the gallery installs and answers a chat request

4. Loading Models

A "model" in LocalAI is a backend engine plus a configuration. You can load models from several sources, all via local-ai run or the web UI.

# From the model gallery (curated catalog at https://models.localai.io)
local-ai run llama-3.2-1b-instruct:q4_k_m

# From Hugging Face (any GGUF file)
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf

# From the Ollama OCI registry
local-ai run ollama://gemma:2b

# From a YAML config file (local path or URL)
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml

# From a standard OCI registry (e.g. Docker Hub)
local-ai run oci://localai/phi-2:latest

Tip

Browse the full catalog with local-ai models list or at models.localai.io. Install from the gallery in the UI under Models, or via the API: POST /models/apply.

Note

Hugging Face downloads honor HF_TOKEN for gated/private repos and HF_ENDPOINT / HF_MIRROR to point at a mirror. Set these as environment variables before starting LocalAI.

Where models live

Models are stored under the path set by --models-path (env LOCALAI_MODELS_PATH), defaulting to ./models. Each model has a <name>.yaml config file (see §8) and its weight files alongside.

5. Everyday Usage: Calling the API

All examples assume the server is at http://localhost:8080. If you've enabled authentication, add -H "Authorization: Bearer YOUR_API_KEY" to every request.

5.1 Chat completions (OpenAI)

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.2-1b-instruct:q4_k_m",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "What is the capital of France?"}
    ],
    "temperature": 0.7,
    "max_tokens": 200
  }'

Stream tokens as they're generated by adding "stream": true; LocalAI returns Server-Sent Events:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama-3.2-1b-instruct:q4_k_m",
       "messages": [{"role":"user","content":"Write a haiku."}],
       "stream": true}'

data: {"choices":[{"delta":{"content":"Silent"}}]}
data: {"choices":[{"delta":{"content":" pond"}}]}
...
data: [DONE]

5.2 Vision (multimodal chat)

Send an image alongside text to a multimodal model:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-vision-model",
    "messages": [{"role": "user", "content": [
      {"type": "text", "text": "What is in this image?"},
      {"type": "image_url", "image_url": {"url": "https://example.com/cat.jpg"}}
    ]}]
  }'

5.3 Embeddings

curl http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model": "your-embedding-model", "input": "The food was delicious."}'

Note

By default embeddings are returned as float arrays. Pass "encoding_format": "base64" to match the OpenAI Node.js SDK default. A model only serves embeddings if its config sets embeddings: true.

5.4 Text-to-speech

curl http://localhost:8080/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model": "your-tts-model", "input": "Hello from LocalAI", "voice": "default"}' \
  --output speech.wav

5.5 Speech-to-text (transcription)

curl http://localhost:8080/v1/audio/transcriptions \
  -F model="whisper-1" \
  -F file="@recording.wav"

For who-spoke-when, use diarization:

curl http://localhost:8080/v1/audio/diarization \
  -F model="your-diarization-model" \
  -F file="@meeting.wav"

5.6 Image generation

curl http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{"model": "your-image-model",
       "prompt": "a red panda coding at a laptop, digital art",
       "size": "512x512"}'

Edit an existing image with inpainting via POST /v1/images/inpainting.

5.7 Reranking

curl http://localhost:8080/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{"model": "your-reranker",
       "query": "What is the capital of France?",
       "documents": ["Paris is the capital of France.", "Berlin is in Germany."]}'

5.8 Tool / function calling

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.2-1b-instruct:q4_k_m",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {"type": "object",
          "properties": {"city": {"type": "string"}}, "required": ["city"]}
      }
    }]
  }'

LocalAI can constrain the model's output so the tool call is always valid JSON (see Constrained output).

5.9 Using the Anthropic and Ollama dialects

Anthropic Messages API:

curl http://localhost:8080/v1/messages \
  -H "Content-Type: application/json" \
  -d '{"model": "llama-3.2-1b-instruct:q4_k_m",
       "max_tokens": 256,
       "messages": [{"role": "user", "content": "Hello!"}]}'

Ollama-compatible chat:

curl http://localhost:8080/api/chat \
  -d '{"model": "llama-3.2-1b-instruct:q4_k_m",
       "messages": [{"role": "user", "content": "Hello!"}]}'

Tip

If you want LocalAI to behave like a drop-in Ollama server (health check on /), start it with --ollama-api-root-endpoint. The /api/* Ollama endpoints are always available regardless of that flag.

6. Working with the Web UI

LocalAI ships a built-in web interface (a React app) served at the root URL. From it you can:

• Chat with any loaded model (multiple chats with saved history).
• Browse and install models from the gallery, with VRAM estimates before you install.
• Browse and install backends on the fly from the backend gallery.
• Import a model by URL (Hugging Face, OCI, YAML).
• Edit model configuration in a YAML editor with autocomplete.
• Manage agents, run them, and watch their reasoning stream live.
• Monitor backends, view backend logs, and see usage metrics.
• Fine-tune and quantize models from the UI (where backends are installed).

Open it at:

http://localhost:8080

Note

The UI talks to the same API documented here. If you prefer a headless server, disable it with --disable-webui — the API endpoints remain available.

Tip

Interactive API documentation (Swagger) is available at http://localhost:8080/swagger/ when the server is running.

7. Feature Walkthroughs

7.1 The model gallery and backend gallery

LocalAI separates models (weights + config) from backends (the engine that runs them). Both come from galleries — catalogs defined as JSON lists of {name, url}.

The default galleries are:

// models  (env LOCALAI_GALLERIES)
[{"name":"localai", "url":"github:mudler/LocalAI/gallery/index.yaml@master"}]
// backends (env LOCALAI_BACKEND_GALLERIES)
[{"name":"localai", "url":"github:mudler/LocalAI/backend/index.yaml@master"}]

List, install, and manage backends from the CLI:

local-ai backends list                 # show available + installed backends
local-ai backends install <backend>    # install one from the gallery
local-ai backends upgrade               # upgrade all (or name them)
local-ai backends uninstall <backend>  # remove one

LocalAI supports 36+ backends and auto-detects the right one for your hardware. Backends are OCI images pulled on demand; you only download the engines you use.

Tip

Enable --auto-upgrade-backends to keep backend images current automatically, or run local-ai backends upgrade on your own schedule.

7.2 Built-in agents

LocalAI includes autonomous AI agents (powered by the LocalAGI/cogito engine) that can use tools, query a knowledge base (RAG), connect to MCP servers, and stream their reasoning. Manage them through the web UI or the API:

You can also run an agent standalone, without the full server:

# Run an agent from the pool registry by name, send one prompt, print the answer
local-ai agent run my-agent --prompt "Summarize today's news"

# Or load an agent from a JSON config file
local-ai agent run --config ./agent.json --prompt "Hello"

# List the agents in your pool registry (agents/pool.json)
local-ai agent list

Note

Standalone agents read from agents/pool.json by default (override with --state-dir). They connect back to a LocalAI API for inference — set --api-url and --api-key if the API isn't the local default. Browse and download shareable agent configs from the Agent Hub.

7.3 Model Context Protocol (MCP)

LocalAI can act as an MCP client — letting a model call tools exposed by external MCP servers — and as an MCP host. MCP-augmented chat has dedicated endpoints:

curl http://localhost:8080/v1/mcp/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "your-model", "messages": [{"role":"user","content":"List my files"}]}'

Discover what a model's configured MCP servers expose:

curl http://localhost:8080/v1/mcp/servers/your-model
curl http://localhost:8080/v1/mcp/prompts/your-model
curl http://localhost:8080/v1/mcp/resources/your-model

MCP servers are declared per model in the model YAML under the mcp: block (see §8). Disable MCP entirely with --disable-mcp.

7.4 Constrained output (grammars)

To force a model to emit only valid structured output (e.g. always-valid JSON for tool calls), LocalAI supports constrained grammars. Provide a grammar in the request or model config, or use response_format for JSON. This is what makes function calling reliable on local models. The grammar field is part of the model's LLM config and can also be set per request.

7.5 Realtime speech-to-speech

The Realtime API provides bidirectional audio with tool calling over a WebSocket:

GET /v1/realtime
POST /v1/realtime/sessions
POST /v1/realtime/transcription_session

This enables low-latency voice assistants where the model listens and speaks, optionally calling tools mid-conversation.

7.6 Voice and face recognition

LocalAI can verify, analyze, register, and identify both faces and voices:

# Register a face under an identity
curl http://localhost:8080/v1/face/register -F model="your-face-model" -F file="@alice.jpg" -F name="alice"
# Identify a face
curl http://localhost:8080/v1/face/identify -F model="your-face-model" -F file="@unknown.jpg"

Parallel endpoints exist for voice: /v1/voice/register, /v1/voice/identify, /v1/voice/verify, /v1/voice/analyze, /v1/voice/embed, /v1/voice/forget.

7.7 Voice activity detection (VAD)

Detect speech segments in audio with Silero-VAD:

curl http://localhost:8080/v1/vad \
  -H "Content-Type: application/json" \
  -d '{"model": "silero-vad", "audio": [/* PCM samples */]}'

7.8 Fine-tuning and quantization

From the UI (or the API) you can fine-tune models with TRL and quantize them to GGUF:

Quantization accepts a Hugging Face model name or local path, a backend (llama-cpp-quantization), and a quantization type (q4_k_m, q5_k_m, q8_0, f16, …), producing a GGUF you can then import and serve.

8. Configuring a Model (YAML)

Most model behavior is controlled by a per-model YAML file in your models directory. The simplest config just names the model and its backend:

name: my-llama
backend: llama-cpp
parameters:
  model: llama-3.2-1b-instruct.Q4_K_M.gguf

A richer example showing common options:

name: my-llama
backend: llama-cpp
f16: true
threads: 8
context_size: 4096
gpu_layers: 35
mmap: true
embeddings: false

parameters:           # request defaults (the OpenAI-style "parameters" block)
  temperature: 0.7
  top_p: 0.9
  top_k: 40
  max_tokens: 512
  seed: -1

template:
  chat: |
    {{.Input}}
  completion: |
    {{.Input}}

stopwords:
  - "</s>"
  - "<|im_end|>"

8.1 Top-level model keys

8.2 The parameters: block (request defaults)

These map to OpenAI request fields and set defaults for the model. Clients can override any of them per request.

8.3 Inline LLM options (llama.cpp / vLLM / transformers)

These live at the top level of the config and apply to most LLM backends:

vLLM-specific keys include gpu_memory_utilization, tensor_parallel_size, max_model_len, dtype, quantization, trust_remote_code, enforce_eager, swap_space, limit_mm_per_prompt, and engine_args (a raw passthrough to the engine constructor).

8.4 The template: block

LocalAI applies Go templates to format prompts for each backend. Set chat, chat_message, completion, edit, function, and multimodal templates, or set use_tokenizer_template: true to use the model's own chat template (common for vLLM and transformers). reply_prefix prepends text to every reply.

8.5 Specialized config blocks

Tip

You rarely need to write a full YAML by hand. Installing from the gallery generates a working config, and the web UI's Edit Model page gives you a YAML editor with autocomplete and VRAM estimation.

9. The local-ai Command Line

local-ai is the single binary. Running it with no subcommand runs the server (the run command). See local-ai <command> --help for full per-command details.

9.1 Common run flags

Every flag has an environment-variable equivalent (shown in parentheses), so you can configure LocalAI entirely through the environment — ideal for containers.

Run with a model on the command line:

local-ai run --address :9090 --threads 8 --context-size 8192 llama-3.2-1b-instruct:q4_k_m

Tip

LocalAI auto-loads environment variables from .env, localai.env, ~/localai.env, ~/.config/localai.env, and /etc/localai.env at startup. Put your LOCALAI_* settings there instead of a long command line.

9.2 CLI for one-off media tasks

# Text to speech without the server
local-ai tts --help

# Audio to text without the server
local-ai transcript --help

9.3 Shell completion

local-ai completion bash > /etc/bash_completion.d/local-ai
local-ai completion zsh  > "${fpath[1]}/_local-ai"
local-ai completion fish > ~/.config/fish/completions/local-ai.fish

10. Common Tasks (Recipes)

Run fully on CPU

Omit any GPU image/flags. LocalAI runs on CPU by default; just don't set --f16/gpu_layers and use a CPU image.

Use LocalAI from the OpenAI Python SDK

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
resp = client.chat.completions.create(
    model="llama-3.2-1b-instruct:q4_k_m",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

Preload models at startup

docker run -ti -p 8080:8080 \
  -e LOCALAI_PRELOAD_MODELS='[{"id":"localai@llama-3.2-1b-instruct:q4_k_m"}]' \
  localai/localai:latest

Keep a model resident in memory

local-ai run --load-to-memory llama-3.2-1b-instruct:q4_k_m llama-3.2-1b-instruct:q4_k_m

Limit how many backends stay loaded

# Keep at most 2 backends in memory; least-recently-used are evicted
local-ai run --max-active-backends 2

Require an API key

local-ai run --api-key "sk-my-secret-key"

Then call with -H "Authorization: Bearer sk-my-secret-key".

Free GPU memory when idle

local-ai run --watchdog-idle --watchdog-idle-timeout 10m

11. Troubleshooting

Troubleshooting entries follow the format Error → Cause → Solution → Prevention.

Error: container name already in use

Cause: You ran docker run --name local-ai ... a second time; the name is still taken by the stopped container. Solution: Restart the existing container instead:

docker start -i local-ai

Prevention: Use docker start to resume, and only docker run for first creation.

Error: macOS says the app "cannot be opened" / is damaged

Cause: The DMG is not signed by Apple, so Gatekeeper quarantines it. Solution:

sudo xattr -d com.apple.quarantine /Applications/LocalAI.app

Prevention: Run the xattr command immediately after installing.

Error: model responds but the first request is very slow

Cause: The matching backend image is downloaded on demand the first time a model type is used. Solution: Wait for the backend to finish downloading (watch the logs). Subsequent requests are fast. Prevention: Preload models/backends at startup, or pre-install the backend with local-ai backends install <backend>.

Error: GPU is not being used

Cause: Wrong image (CPU image instead of a GPU one), missing device flags, or gpu_layers/--f16 not set. Solution: Use the GPU image for your hardware (see §3.1), pass the device flags, and set gpu_layers in the model config. Issue #8934 tracks an Intel-GPU detection case. Prevention: Verify --gpus all (NVIDIA) or the --device flags (AMD/Intel) are present and the backend matches your accelerator.

Error: embeddings return the wrong number of dimensions

Cause: Some embedding models (e.g. Jina-v3) need configuration to return full-dimension vectors; see issue #8721. Solution: Check the model's gallery config/documentation for the correct embedding settings, and ensure embeddings: true is set on a model intended for embeddings. Prevention: Validate embedding dimensions against the model card before using them in a vector store.

Error: diffusers / stable-diffusion image generation fails

Cause: Known backend regressions in some versions; see issue #9979. Solution: Update LocalAI and the diffusers backend (local-ai backends upgrade), and check the backend logs in the UI. Prevention: Keep backends current; test image generation after upgrades.

Error: server refuses to start when binding to a public address

Cause: LocalAI refuses to listen on a public-internet address with no authentication configured (a safety guard). Loopback, RFC 1918 LAN, ULA, link-local, and CGNAT/Tailscale ranges are allowed. Solution: Configure authentication (set --api-key or enable auth), or — only if you understand the risk — pass --allow-insecure-public-bind. Prevention: Always put auth in front of a publicly bound server.

Error: out of memory / system RAM misreported in containers

Cause: In some container runtimes (e.g. LXD) system RAM is reported incorrectly; see issue #8059. Solution: Set context_size, gpu_layers, and --max-active-backends conservatively; enable the memory reclaimer (--memory-reclaimer). Prevention: Size models to your hardware and cap concurrent backends.

Error: web UI logs are noisy / health checks flood the log

Cause: UI polling and health-check paths log at INFO by default in older builds. Solution: Update LocalAI; polling/health/resource paths now log at DEBUG (hidden by default). Prevention: Keep LocalAI current.

Tip

When in doubt, check the per-backend logs in the web UI (or GET /api/backend-logs/:modelId) — backend startup failures (missing libraries, GLIBCXX mismatches, etc.) show up there, not in the main server log.

12. FAQ

Q: Do I need a GPU? No. LocalAI runs on CPU-only hardware. A GPU (NVIDIA, AMD, Intel, Apple Silicon, or Vulkan) accelerates inference but is optional.

Q: Is it really compatible with the OpenAI API? Yes — it implements the OpenAI chat, completions, embeddings, audio, and images endpoints, plus Anthropic, Ollama, ElevenLabs, and Jina dialects. Point existing SDKs at LocalAI by changing the base URL.

Q: Where do models come from? The curated LocalAI gallery (local-ai models list / models.localai.io), Hugging Face (huggingface://), the Ollama registry (ollama://), or any OCI registry (oci://).

Q: Why is the download so big the first time? The core is small; each backend engine is a separate image pulled on demand. The first use of a new model type downloads that backend once.

Q: How do I run more than one model? Load several; LocalAI manages them. Use --max-active-backends to cap how many stay in memory at once (least-recently-used are evicted).

Q: Can I keep data out of the cloud entirely? Yes — LocalAI is privacy-first and self-hosted; your data never leaves your infrastructure. The only network calls are to fetch models/backends you explicitly request.

Q: How do I require authentication? Start with --api-key for static keys, or enable full user auth (OIDC/GitHub/local) for multi-user setups (see the Admin Guide).

Q: Does it support streaming? Yes — add "stream": true to chat/completions requests to receive Server-Sent Events.

Q: Can the model call tools/functions? Yes — pass tools in the request. LocalAI can constrain output with grammars so tool-call JSON is always valid.

Q: What's an "agent" vs. a model? A model answers single requests. An agent (LocalAGI/cogito) is autonomous — it plans, uses tools, queries a knowledge base, and can connect to MCP servers. Run agents from the UI, the API, or standalone with local-ai agent run.

Q: How do I update? Pull the latest image (docker pull localai/localai:latest) and recreate the container, or local-ai backends upgrade to refresh backends.

Q: Can I disable the web UI? Yes — --disable-webui. The API stays available.

Q: Where are generated images/audio saved? Under the path set by LOCALAI_GENERATED_CONTENT_PATH (default /tmp/generated/content).

Q: How do I free GPU memory when models are idle? Enable the idle watchdog (--watchdog-idle --watchdog-idle-timeout 10m) or the memory reclaimer (--memory-reclaimer).

Q: Can I run it on a Mac? Yes — download the signed-but-unnotarized DMG and clear the quarantine attribute (see §3.2). Apple Silicon is supported via MLX and llama.cpp backends.

Q: How do I see what the API offers interactively? Open http://localhost:8080/swagger/.

13. Reference

Selected API endpoints

Common environment variables

Model URI schemes

Useful links

• Documentation: https://localai.io/
• Model catalog: https://models.localai.io/
• Agent Hub: https://agenthub.localai.io
• Source: https://github.com/mudler/LocalAI
• Community: Discord