Gradio Admin & Architecture Guide

This guide is for administrators, DevOps engineers, platform architects, and contributors who need to deploy, secure, operate, scale, or extend Gradio — and to understand how it works internally. For end-user, app-building documentation, see the companion User Guide.

Gradio is a full-stack framework distributed as a single Python package (gradio, version 6.x) but composed of a Python backend, a Svelte/TypeScript frontend, and two client libraries (Python and JavaScript) that all live in one monorepo. This document covers each subsystem in turn.

1. Executive Summary

Operational headline: A Gradio process is a single ASGI application. It serves the SPA, a configuration endpoint, a file-serving route, a Server-Sent-Events queue for function execution, and (optionally) an MCP endpoint. There is no database and no external broker — session state lives in memory. Horizontal scaling therefore requires sticky sessions or a stateless app design.

2. System Architecture

                          ┌─────────────────────────────────────────────┐
   Browser / Client       │             Gradio Process (ASGI)           │
 ┌──────────────────┐     │                                             │
 │  SPA (Svelte)    │◀───▶│  FastAPI App  (gradio/routes.py:App)        │
 │  @gradio/client  │     │   ├── GET  /                → SPA HTML       │
 │  gradio_client   │     │   ├── GET  /config          → app config    │
 │  MCP client      │     │   ├── POST /login, /logout  → auth          │
 │  AI agent        │     │   ├── GET  /file={path}     → file serving  │
 └──────────────────┘     │   ├── POST /queue/join (SSE)→ execution     │
                          │   ├── /gradio_api/mcp/*     → MCP server    │
                          │   └── /static, /assets, …   → frontend      │
                          │                                             │
                          │  Queue (queueing.py)   ── concurrency ──▶ fn │
                          │  StateHolder (in-memory per-session state)  │
                          │  Theme engine · Examples cache · Telemetry  │
                          └───────────────┬─────────────────────────────┘
                                          │ (optional, ssr_mode)
                                  ┌───────▼────────┐
                                  │  Node.js proxy │  server-side rendering
                                  │  (SSR / static │  + static-asset workers
                                  │   workers)     │
                                  └────────────────┘

Data flow of a single prediction:

1. The browser SPA loads /config, which describes every component, layout, and event dependency as JSON.
2. A user action triggers an event. The SPA opens an SSE connection to the queue (/queue/join-style route).
3. The Queue assigns the event a position, respects concurrency_limit/concurrency_id, and when a slot frees, preprocesses inputs (component .preprocess()), runs the Python fn, and postprocesses outputs (component .postprocess()).
4. Results (or streamed chunks) are pushed back over SSE and rendered.

Note

The single most important architectural fact for operators: the queue is an in-process, in-memory component, and session state (gr.State) lives in process memory (StateHolder, capped by state_session_capacity, default 10,000 sessions). This shapes every scaling and deployment decision below.

3. Repository Structure (Monorepo)

The repository is a polyglot monorepo. The published gradio PyPI package is built from gradio/ plus bundled frontend assets.

Caution

A home/runner/work/gradio/gradio/ subtree may appear in checkouts/artifacts — this is a CI build-path duplicate, not real source. The authoritative source trees are gradio/, js/, and client/. Ignore the home/runner/... mirror when navigating the codebase.

4. The Python Backend (gradio/)

The backend is a single Python package. Key modules and their responsibilities:

4.1 The Block hierarchy

Block                       (base: id, visibility, elem_id/classes, file caching)
 └─ BlockContext            (containers that hold children: layouts)
     └─ Blocks              (the app root; context manager; launch/queue)
         ├─ Interface
         ├─ TabbedInterface
         └─ ChatInterface

Every component and layout is a Block. A Blocks app holds a BlocksConfig (self.default_config) containing the blocks dict (id → Block) and the fns dict (id → BlockFunction). BlocksConfig.get_config() serializes the whole app to the JSON the frontend consumes via /config.

4.2 Event registration

BlocksConfig.set_event_trigger(...) is the central event-wiring method. Each .click()/.change()/etc. listener method ultimately calls it, producing a BlockFunction with all execution metadata: inputs, outputs, preprocess/postprocess, queue, batch/max_batch_size, concurrency_limit/concurrency_id, api_name/api_visibility, trigger_mode, cancels, streaming settings, and an optional validator.

Note

api_visibility ("public"/"private"/"undocumented") replaced the older show_api boolean. When loading old configs, from_config() migrates show_api/api_name=False into the new field.

5. The Frontend (js/ Svelte Monorepo)

The frontend is a pnpm workspace (pnpm-workspace.yaml) of ~50 packages built with Vite and Svelte, with Storybook for isolated component development.

Backend ↔ frontend binding: A Python component's get_block_name() returns a lowercased class name that selects the matching Svelte component package on the frontend. Custom components follow the same convention, which is why the toolchain can scaffold both halves together.

Build artifacts: The published Python wheel bundles compiled frontend assets (gradio/templates, gradio/_frontend_code, gradio/node) so end users never run a JS toolchain — pip install gradio ships a ready-to-serve UI.

Tip

When contributing, frontend changes require building the JS workspace (pnpm i && pnpm build) and copying assets into the Python package; the repo's .config/copy_frontend.py automates this for source installs. End users and operators never need Node.js unless they enable ssr_mode.

6. The Client Libraries

Gradio auto-generates a typed API from every app. Two official clients consume it.

6.1 Python client (gradio_client)

client/python/gradio_client/client.py exposes:

6.2 JavaScript client (@gradio/client)

client/js/ provides Client.connect(...) with .predict(endpoint, payload) and streaming support, used both by external apps and internally by the SPA.

Note

The backend and the JS client share types: the same config that drives the UI drives the client, so client calls stay in lock-step with the app's actual endpoints.

7. Request Lifecycle & the Queue

queueing.py implements the concurrency engine. It is enabled by default (Blocks.__init__ calls self.queue()).

7.1 Core objects

7.2 Execution path

1. push() — an incoming event is validated and enqueued; the client receives a position estimate.
2. start_processing() — the queue loop pulls events as slots free up under the relevant concurrency_id's limit.
3. process_events() — preprocess → run fn (in a worker thread or async) → postprocess → stream results back over SSE; handles batching, cancellation (cancels), and gr.Progress updates.

7.3 Concurrency controls

Important

On Hugging Face ZeroGPU Spaces, the queue forces max_size=1 automatically. Plan capacity around per-event concurrency_id pools rather than raw thread count when GPU-bound.

8. Server Mode (gr.Server)

gr.Server (server.py) is a FastAPI subclass that exposes Gradio's API engine (queue, SSE streaming, concurrency, MCP, Spaces hosting) without a Gradio UI. Use it to build a custom frontend on Gradio's backend.

from gradio import Server

server = Server()

@server.api(name="hello")          # Gradio-managed endpoint (queue + streaming)
def hello(name: str) -> str:
    return f"Hello {name}"

@server.get("/health")              # ordinary FastAPI route
def health():
    return {"ok": True}

@server.mcp.tool                    # tag as an MCP tool
def add(a: int, b: int) -> int:
    return a + b

server.launch()

Because Server is a FastAPI app, all standard FastAPI methods (.get, .post, .add_middleware, .include_router) work directly. The server.mcp namespace provides .tool(), .resource(), and .prompt() decorators.

9. MCP Server Subsystem

mcp.py (GradioMCPServer) turns a Gradio app into a Model Context Protocol server so AI agents can call your functions as tools.

• Enable: demo.launch(mcp_server=True) or set GRADIO_MCP_SERVER=True. Requires the [mcp] extra.
• Tool surface: every public, documented function (one with an api_name and a docstring/typed signature) becomes an MCP tool. Visibility honors api_visibility.
• Endpoints: the MCP server is mounted under a sub-path of the app; a browser GET to the MCP endpoint returns a human-readable landing page.
• Server mode: gr.Server adds explicit mcp.tool/mcp.resource/mcp.prompt decorators for fine-grained registration.

Tip

Type hints and docstrings are not optional for MCP — they generate the tool schema the agent reads. Document every function you intend to expose.

10. Configuration Reference

Gradio is configured through launch()/queue() parameters and environment variables. Environment variables are convenient for containerized deployments where you cannot edit code.

10.1 Core server variables

10.2 File-serving, cache & security variables

10.3 Behavior & integration variables

Note

launch() parameters always win over environment variables when both are set. Many launch() arguments document their matching env var inline (e.g. share ↔ GRADIO_SHARE).

11. Security

Gradio's security model centers on file serving, authentication, request origin, and outbound request safety. (SECURITY.md: report vulnerabilities privately to [email removed].)

11.1 File-serving guards

User-facing components serve files through the /file={path} route, which is gated by:

• login_check dependency — file routes require authentication when auth is enabled.
• Path allow/block lists — files must pass is_in_or_equal() against the working directory, the temp cache, and allowed_paths. blocked_paths takes precedence over everything, including Gradio's own defaults.

Warning

Be deliberate with allowed_paths: adding a directory exposes all files in it and its subdirectories to every user of the app. Never add /, a home directory, or a secrets directory. Use blocked_paths to fence off sensitive subtrees.

11.2 Authentication

auth and auth_dependency are mutually exclusive. Authenticated routes (/config, /file=, etc.) enforce a login_check dependency.

11.3 Request origin (CORS / CSRF)

CustomCORSMiddleware is installed with strict_cors=True by default, blocking cross-origin requests to a localhost server (CSRF protection). Set strict_cors=False only when embedding a locally running app via web components, and understand the trade-off.

11.4 Outbound-request safety (SSRF)

• Remote file downloads go through safehttpx / processing_utils.async_ssrf_protected_download(), which blocks requests to internal/metadata addresses.
• When loading remote apps via gr.load()/from_config(), component proxy_urls are restricted to *.hf.space hosts to prevent SSRF via a malicious config.

11.5 Upload limits & transport

11.6 Hardening checklist

• [ ] Add auth= or auth_dependency= for any non-public app
• [ ] Set an explicit max_file_size (e.g. "10mb") if you accept uploads
• [ ] Audit allowed_paths; add blocked_paths for any sensitive subtree
• [ ] Keep strict_cors=True unless embedding locally
• [ ] Run unprivileged (non-root) behind a reverse proxy with TLS termination
• [ ] Never enable --vibe mode on a reachable server — it permits arbitrary code execution
• [ ] Set GRADIO_ANALYTICS_ENABLED=False if telemetry is disallowed by policy
• [ ] Pin the gradio version and review the changelog before upgrading (the API surface evolves between majors)

Caution

Vibe mode (gradio --vibe) lets anyone who reaches the endpoint modify host files and execute arbitrary code. It is a local development tool only. The CLI prints a warning, but operators must ensure it is never exposed.

12. Deployment & Operations

12.1 Deployment options

Hugging Face Spaces (recommended)

gradio deploy

Or create a Space with the Gradio SDK and push app.py + requirements.txt. Spaces auto-enables example caching, PWA, and (optionally) a Duplicate button.

Self-hosted (ASGI)

Run behind any ASGI server / process manager. For multiple workers or a reverse proxy, set root_path to the mount point:

demo.launch(server_name="0.0.0.0", root_path="/myapp")

Docker

FROM python:3.12-slim
RUN pip install gradio
COPY app.py .
ENV GRADIO_SERVER_NAME=0.0.0.0
EXPOSE 7860
CMD ["python", "app.py"]

Google Cloud Run

gradio deploy   # choose the Cloud Run target

12.2 Reverse proxy

When fronting Gradio with nginx/Traefik/etc., set root_path (or GRADIO_ROOT_PATH) to the sub-path and ensure the proxy forwards WebSocket/SSE upgrade headers — the queue depends on Server-Sent Events.

12.3 Operating as a process

12.4 Multi-instance scaling

Because state and the queue are in-memory, scaling out requires either:

• Stateless apps (no gr.State that must persist across requests) behind a load balancer, or
• Sticky sessions so a user's SSE connection and state stay on one instance.

For GPU-bound workloads, prefer a single instance with concurrency_id pools over many instances competing for the same hardware.

13. Performance & Scaling

Tip

Streaming (generator fns) improves perceived latency without changing throughput. Combine with a sensible concurrency_limit so long-running streams don't starve the queue.

14. Build System & Release

Runtime dependencies (from requirements.txt): fastapi, starlette, uvicorn, pydantic (v2), httpx, huggingface_hub, hf-gradio, gradio_client, safehttpx, pandas, numpy, pillow, pydub, orjson, brotli, Jinja2/markupsafe, typer, anyio, python-multipart, pyyaml, tomlkit, semantic_version, groovy.

Note

gradio_client is pinned exactly (==2.5.0) — the client and server share a wire protocol, so they are released in lock-step. When upgrading gradio, let it pull the matching gradio_client.

15. Code Quality & Testing

Conventions: components are paired Python+Svelte packages; the Python side handles pre/postprocessing and config serialization, the Svelte side handles rendering and events. New components should follow this dual-package convention and ship .pyi stubs and a Storybook story.

16. Extending Gradio

16.1 Custom components

The gradio cc toolchain scaffolds, develops, builds, and publishes custom components (Python backend + Svelte frontend):

gradio cc create MyComponent     # scaffold
gradio cc dev                    # hot-reload dev server
gradio cc build                  # build for distribution
gradio cc publish                # publish to PyPI + the HF gallery

A custom component is a normal Block subclass on the Python side plus a Svelte Index.svelte on the frontend, bound by the get_block_name() naming convention.

16.2 Themes

Subclass gr.themes.Base (or Soft, Monochrome, Glass, Origin, Citrus, Ocean) to define colors, fonts, and sizing. Build interactively with gr.themes.builder() and publish to the Hub with the upload_theme entry point. Apply with demo.launch(theme=...).

16.3 Loading remote apps

gr.load("user/space") reconstructs a remote Gradio app locally from its config (via Blocks.from_config()), letting you compose or re-host existing Spaces. gr.load_chat() and gr.load_openapi() wrap chat endpoints and OpenAPI specs respectively.

17. Operational Troubleshooting

Symptom: Queue/SSE stalls behind a reverse proxy

Cause: The proxy buffers or drops Server-Sent-Events / upgrade headers. Fix: Disable response buffering for the Gradio location and forward Connection/Upgrade headers; set root_path to the mount point.

Symptom: Files 404 or "not allowed" in production

Cause: Served file is outside the allowed directories (is_in_or_equal check). Fix: Add its directory to allowed_paths (or GRADIO_ALLOWED_PATHS); confirm it isn't caught by blocked_paths.

Symptom: App unreachable from other hosts

Cause: Default bind is 127.0.0.1. Fix: server_name="0.0.0.0" / GRADIO_SERVER_NAME=0.0.0.0, and open the port.

Symptom: SSR errors or blank first paint

Cause: ssr_mode=True without Node 20+, or SSR with a multi-page app mounted on FastAPI (unsupported — Gradio disables SSR with a warning). Fix: Install Node 20+, or set ssr_mode=False.

Symptom: Memory grows over time

Cause: Accumulated session state and cached temp files. Fix: Lower state_session_capacity; set delete_cache=(freq, age); ensure gr.State objects aren't unbounded.

Symptom: ORJSON/serialization error with api_name=False as a dict key

Cause: Known issue (#13482). Fix: Track the upstream issue; avoid using api_name=False as a dictionary key.

Symptom: Port exhaustion on restart

Cause: Previous process didn't release the port; range 7860–7959 occupied. Fix: gr.close_all() on shutdown; set an explicit server_port.

Symptom: Unexpected outbound network calls at startup

Cause: Telemetry + version check. Fix: GRADIO_ANALYTICS_ENABLED=False.

18. Quick Reference

Architecture at a glance

Critical operational defaults

Security one-liners

demo.launch(
    server_name="0.0.0.0",
    auth=("admin", "secret"),
    max_file_size="10mb",
    blocked_paths=["/etc", "/root"],
    ssl_keyfile="key.pem", ssl_certfile="cert.pem",
)

Endpoints (selected)