Sentry Administrator & Architecture Guide

This guide is for the engineers, architects, and operators who deploy, operate, secure, and extend Sentry — the getsentry/sentry server that powers both sentry.io and self-hosted installations. It documents the system architecture, the service topology, the complete configuration and CLI surface, the security model, day-2 operations, and the internals a developer needs to work in the codebase.

For the product/feature walkthrough and SDK onboarding, see the companion User Guide.

Note

Sentry is a large, polyglot, multi-service platform — roughly 20,000 files across a Django/Python backend and a React/TypeScript frontend, fronting a fleet of supporting services (PostgreSQL, Redis, Kafka, ClickHouse/Snuba, Relay, Symbolicator). This guide maps that system from the source as it exists at commit cd5d5048. Where a subsystem is implemented across many files, the guide names the authoritative module so you can read further.

1. Executive Summary

Sentry ingests application telemetry (errors, traces/spans, replays, profiles, logs, session health, cron check-ins, uptime probes), processes it through a streaming pipeline, stores it across purpose-built backends, and serves a React single-page application plus a public REST API for triage and analysis.

2. System Architecture

2.1 The event pipeline

The end-to-end path of a single error event:

   Your application
        │  (SDK: sentry-python, sentry-javascript, …)
        ▼
   ┌──────────┐   envelope over HTTPS
   │  Relay   │   • inbound filters, PII scrubbing, quotas/rate limits,
   │  (Rust)  │     normalization, dynamic sampling
   └────┬─────┘
        │ publishes to Kafka topics
        ▼
   ┌──────────────────── Kafka ────────────────────┐
   │  ingest-events, ingest-transactions, replays,  │
   │  outcomes, attachments, …                       │
   └────┬───────────────────────────────────────────┘
        │  consumed by Arroyo-based consumers
        ▼
   ┌─────────────────┐      ┌──────────────────────────┐
   │ sentry consumers │ ──▶ │ Symbolicator (native/JS)  │
   │ (ingest, post-   │     └──────────────────────────┘
   │  process-forward,│
   │  snuba writers)  │ ──▶ Nodestore (raw event blob)
   └───────┬─────────┘ ──▶ PostgreSQL (Group/Issue, metadata)
           │            ──▶ Snuba ▶ ClickHouse (searchable events, aggregates)
           ▼
   ┌──────────────────┐   post-processing: grouping, alert rule
   │ taskworker tasks │   evaluation, notifications, integrations,
   │ (gRPC taskbroker)│   suspect commits, Seer triggers
   └──────────────────┘
           │
           ▼
   Notifications → email / Slack / Teams / Discord / PagerDuty / Opsgenie / webhooks

Reading the diagram:

1. Ingestion (Relay). SDKs send envelopes to Relay. Relay applies inbound filters, server-side PII scrubbing, rate limits/quotas, normalization, and dynamic sampling, then publishes to Kafka. SENTRY_USE_RELAY = True in the self-hosted settings.
2. Streaming (Kafka + Arroyo). Sentry's consumers (sentry run consumer <name>) read Kafka topics. The Arroyo library (sentry-arroyo) provides the consumer runtime with batching, offset management, dead-letter queues (DLQ), and health checks.
3. Storage fan-out. Events are written to Nodestore (raw blob), PostgreSQL (the Group/issue and relational metadata), and — via Snuba — to ClickHouse, which backs all search, stats, and aggregation.
4. Post-processing (taskworker). Grouping, alert-rule evaluation, digest building, integration syncs, suspect-commit resolution, and Seer triggers run as background tasks executed by sentry run taskworker against a gRPC taskbroker, scheduled by sentry run taskworker-scheduler.
5. Serving. The Django/DRF app (run by Granian via sentry run web) serves the REST API and the React SPA; reads come from PostgreSQL, Redis, and Snuba.

Important

Errors can group and appear from PostgreSQL even when Snuba is down, but search, stats, dashboards, Discover, and metric alerts all depend on Snuba/ClickHouse. Treat Snuba and the Kafka consumers as load-bearing for the analytics half of the product.

2.2 Hybrid-cloud silo model

To support data-residency (US/EU) and isolation, Sentry runs in a siloed topology controlled by SENTRY_SILO_MODE (src/sentry/conf/server.py):

Cross-silo communication goes through an RPC layer (*/services/ packages expose rpc-decorated service methods; schema validated via sentry rpcschema) and an outbox/replica mechanism (hybridcloud/) that asynchronously replicates control-silo objects (e.g. API tokens) to regions. Self-hosted operators run monolith mode and can ignore silo configuration; SaaS runs split silos with an API Gateway (src/apigw/) routing requests to the correct region.

2.3 Backend service architecture

The Django application is organized as one project with many internal Django apps under src/sentry/. Each major product/subsystem is a package:

2.4 Frontend architecture

The SPA lives in static/app/ (with static/gsApp/ and static/gsAdmin/ for SaaS billing/admin surfaces):

• Stack: React 19, TypeScript 6, rspack bundler (Rust webpack-compatible), Emotion for CSS-in-JS, react-router-dom 6 for routing.
• State: legacy Reflux stores + MobX in places, with TanStack Query (@tanstack/react-query) for server-state/data fetching and nuqs for URL state.
• Entry points: static/app/index.tsx → static/app/main.tsx; routes in static/app/router/; views in static/app/views/; shared components in static/app/components/.
• Build: pnpm workspace; rspack.config.ts at the root. Dev is served by sentry devserver (which also proxies to the backend) or pnpm dev-ui.

Note

The frontend includes onboarding doc generators (static/app/gettingStartedDocs/) with one module per platform — these render the per-SDK install snippets you see in the project-creation flow.

3. Requirements

Runtime dependencies (services)

A functioning Sentry needs all of:

Application runtime

Caution

Sentry pins Pydantic v1 (pydantic>=1.10.26,<2). When working in the codebase, do not assume Pydantic v2 APIs.

4. Installation & Deployment

4.1 Self-hosted (recommended path)

Use the official Compose project, which wires up Sentry plus every required service:

git clone https://github.com/getsentry/self-hosted.git
cd self-hosted
./install.sh          # interactive: builds images, runs migrations, creates the first user
docker compose up -d  # start the full stack

install.sh runs sentry upgrade (migrations + Kafka topics + nodestore bootstrap) and prompts you to create the initial owner account. The web UI is served on port 9000.

4.2 The application image

The image built from this repo (self-hosted/Dockerfile) is the unit the Compose project runs for every Sentry role:

• Base python:3.13.1-slim-bookworm, runs as non-root user sentry (uid/gid 999).
• SENTRY_CONF=/etc/sentry, GRPC_POLL_STRATEGY=epoll1.
• Dependencies installed with uv sync --frozen.
• ENTRYPOINT ["/docker-entrypoint.sh"], default CMD ["run", "web"] → sentry run web on port 9000, VOLUME /data.
• The same image is launched with different commands for each role (run web, run taskworker, run consumer …, etc.).

4.3 Configuration files

Configuration lives in $SENTRY_CONF (default /etc/sentry), seeded from self-hosted/:

sentry init <directory> scaffolds a fresh config directory (--dev for development-friendly settings).

4.4 Key deployment environment variables

The self-hosted sentry.conf.py reads these (all optional unless noted):

Important

If you terminate TLS at a reverse proxy, uncomment the proxy settings in sentry.conf.py: SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https'), SESSION_COOKIE_SECURE = True, CSRF_COOKIE_SECURE = True. Otherwise secure cookies and HTTPS detection will be wrong.

4.5 Process topology in production

A real deployment runs these roles (each is the image with a different command):

• [ ] sentry run web — Granian web/API server (scale horizontally behind a proxy).
• [ ] sentry run taskworker — background task workers (scale by replica/concurrency).
• [ ] sentry run taskworker-scheduler — single scheduler for periodic tasks.
• [ ] sentry run consumer <name> — one or more per Kafka topic (ingest, post-process-forward, snuba writers, replays, outcomes, …).
• [ ] Supporting services: PostgreSQL, Redis, Kafka, ClickHouse + Snuba, Relay, Symbolicator (run by the Compose project).

5. Configuration Reference

Sentry has two configuration layers:

1. Django settings in src/sentry/conf/server.py (and your sentry.conf.py overrides) — SENTRY_* Python constants and backend class paths. These are set at process start.
2. Runtime options (SENTRY_OPTIONS / the options store) — key/value options readable and writable at runtime via sentry config and the UI, stored in the DB/cache. Many are also pinnable in config.yml.

5.1 Pluggable backends (set in settings)

The self-hosted defaults wire Sentry's subsystems to Redis/Snuba:

5.2 Selected environment-driven settings

These are read directly from the environment in conf/server.py and elsewhere (non-exhaustive; grep os.environ/os.getenv in conf/server.py for the full list):

5.3 The options store (config.yml / sentry config)

YAML-pinned options use dotted keys. Examples from the shipped config.yml:

# Mail
mail.backend: 'smtp'
mail.host: 'localhost'
mail.port: 25
mail.from: 'root@localhost'

# File storage
filestore.backend: 'filesystem'
filestore.options:
  location: '/data/files'

# System
system.secret-key: '...'   # prefer SENTRY_SECRET_KEY env

# Redis clusters (referenced by name from backends)
redis.clusters:
  default:
    hosts:
      0: {host: 127.0.0.1, port: 6379}

Manage options at runtime:

sentry config list             # list options
sentry config get mail.host    # read one
sentry config set mail.host smtp.acme.com
sentry config delete <option>
sentry configoptions sync -f options.yaml   # bulk apply from a file (automator)

6. Required Secrets & Credentials

Before a production deployment will run cleanly, provision these (env vars or config.yml, ideally from a secrets manager):

Important

Rotating SENTRY_SECRET_KEY invalidates every active session. Rotate deliberately and communicate the forced re-login.

7. The Public REST API (operational view)

Base path /api/0/, served by DRF. The SaaS host is https://{region}.sentry.io (region ∈ us, de); on self-hosted it is your instance URL.

• Auth: Authorization: Bearer <token> — organization auth tokens (preferred for automation), internal-integration tokens (Sentry Apps), or user tokens. Session auth is used by the web UI.
• Schema: generated by drf-spectacular (make build-api-docs); a curated public spec is api-docs/openapi.json (tags: Teams, Organizations, Projects, Events, Releases, Integration, SCIM). The full surface is the 190+ endpoint modules under src/sentry/api/endpoints/ wired in src/sentry/api/urls.py.
• Pagination: cursor-based via the Link header.
• Rate limiting: enforced (sentry.ratelimits); clients must honor 429 + Retry-After.

Note

The AST-extracted api_endpoints.tsv for this repo is not a reliable endpoint list — DRF endpoints are class-based and the extractor instead captured @patch(...) decorators from the test suite. The authoritative sources are api-docs/openapi.json (public subset) and src/sentry/api/urls.py + the endpoints/ package (complete internal + public surface).

See the User Guide §13 for request examples.

8. The sentry CLI — Complete Reference

The CLI is a Click application (src/sentry/runner/, entry runner/main.py), exposed as the sentry console script. Global options: --config <dir>, -l/--loglevel, --logformat. Run sentry help for the live list. Command groups and their members (from runner/commands/):

Server & workers — sentry run, start, upgrade

Identity & access

Configuration

Backup, import & export (relocation)

Data lifecycle & operations

Diagnostics, notifications & developer tools

Tip

On the self-hosted Compose project, run CLI commands with docker compose run --rm web sentry <command> so they execute inside the app environment with the right config and network access.

9. Operations

9.1 Upgrades

1. Stop or drain workers/consumers (web can stay up briefly).
2. Pull the new image / git pull the self-hosted project.
3. Run sentry upgrade (applies Django migrations, bootstraps nodestore, creates Kafka topics with --create-kafka-topics).
4. Restart web, taskworkers, scheduler, and consumers.

The self-hosted project provides an install.sh that performs the upgrade steps in order; prefer it for self-hosted.

Warning

Migrations must complete before the new application code serves traffic. Running a new web image against an un-migrated database will error. Always sentry upgrade first.

9.2 Backups & relocation

Use the export/import command families (§8). For full-fidelity moves, encrypt with --encrypt-with (or GCP KMS) and keep the findings file. sentry backup compare validates two exports; sentry backup sanitize scrubs sensitive fields for sharing.

9.3 Data retention & cleanup

• sentry cleanup --days N prunes old rows in PostgreSQL/nodestore; use --concurrency and partitioned deletion for large installs.
• Event storage in ClickHouse/Snuba is governed by its own TTL/retention — configure retention there for the bulk of event volume.
• The _SENTRY_CLEANUP env flag and --silent support unattended runs.

9.4 Monitoring the platform itself

Watch:

• Kafka consumer lag and DLQ topics (--enable-dlq). Growing lag = ingestion falling behind.
• Snuba/ClickHouse health (search/stats depend on it).
• Taskworker throughput and queue depth; the scheduler must be running for periodic jobs.
• Redis memory (buffers/quotas/digests live here).
• Granian web worker saturation.
• The built-in health surface and backpressure-monitor for self-protection.

9.5 Performance tuning

10. Security

10.1 Authentication & authorization

• Accounts: username/email + password, with optional 2FA (TOTP, WebAuthn/ U2F via fido2/python-u2flib-server).
• SSO/SAML: src/sentry/auth/providers/ ships SAML2 providers (Okta, OneLogin, Rippling, Auth0, generic), plus Google and GitHub auth. SSO session expiry via SENTRY_SSO_EXPIRY_SECONDS. SCIM endpoints provision users/groups.
• Superuser & staff: elevated access is gated by the superuser/staff flags and granular permissions (sentry permissions). _SENTRY_SKIP_CONFIGURATION and staff access map to the Django admin backend.
• API tokens: organization auth tokens, user tokens, and internal-integration tokens, each scope-limited. Tokens are replicated control→region via the outbox mechanism.

10.2 Secrets & encryption

• SENTRY_SECRET_KEY (≥ 32 chars) signs sessions and tokens; rotating it invalidates all sessions. The self-hosted config refuses to boot without it and warns if it is too short.
• Field-level at-rest encryption uses Fernet keys (DATABASE_ENCRYPTION_FERNET_PRIMARY_KEY_ID / _KEYS_LOCATION).
• Webhooks and internal RPC are HMAC-signed (WEBHOOK_SIGNING_SECRET, SENTRY_SUBNET_SECRET, taskworker TASKWORKER_SHARED_SECRET).

10.3 PII & data scrubbing

Defense-in-depth, two layers:

1. Client-side — SDK send_default_pii=False, before_send hooks, and per-SDK scrubbing.
2. Server-side — Relay applies data-scrubbing rules and Enhanced Privacy before storage; project Security & Privacy settings define scrubbing rules, sensitive-field lists, and IP-address storage policy.

Caution

Inconsistent email scrubbing under Enhanced Privacy has been reported (#110838). Do not rely on a single scrubbing layer for regulated data — configure both SDK-side and server-side rules and verify with test events.

10.4 Network & transport

• Put Sentry behind TLS termination and set the proxy SSL header + secure-cookie settings (§4.4).
• CSP is supported via django-csp. CSRF protection is Django's, with secure cookies recommended.
• Restrict Kafka, ClickHouse, PostgreSQL, and Redis to the internal network; only Relay's ingest endpoint and the web port need external exposure.

10.5 Licensing & compliance

Sentry is under FSL-1.1-Apache-2.0 (LICENSE.md): source-available, free to self-host and modify, with a non-compete restriction that lapses two years after each release (then Apache-2.0). Confirm obligations before redistributing or offering Sentry-as-a-service.

11. Developer Internals

11.1 Repository layout

src/sentry/          Backend Django project (api, integrations, issues, models, …)
src/sentry/conf/     server.py — canonical settings + SENTRY_OPTIONS
src/sentry/runner/   The `sentry` CLI (commands/)
src/apigw/           SaaS API gateway (region routing)
src/flagpole/        Feature-flag evaluation engine
src/bitfield/, src/sudo/, src/social_auth/, src/django_picklefield/   vendored/support libs
static/app/          React SPA (views/, components/, router/, stores/)
static/gsApp/, static/gsAdmin/   SaaS billing & internal admin UIs
api-docs/            OpenAPI spec + build tooling
tests/               Python + JS + acceptance tests
migrations_lockfile.txt   Guards migration ordering
devservices/, devenv/     Local development service orchestration
self-hosted/         Dockerfile + default config for the self-hosted image

11.2 Local development

# Bootstrap the dev environment (devenv + devservices)
make bootstrap            # or: devenv sync
# Run the all-in-one dev server (backend + frontend proxy)
sentry devserver --workers
# Frontend-only hot reload
pnpm dev-ui

devservices brings up the supporting containers (PostgreSQL, Redis, Kafka, ClickHouse/Snuba, Relay, Symbolicator) locally. sentry devserver runs the web app, optionally ingest (--ingest), workers (--workers), and the task scheduler (--task-scheduler).

11.3 Adding a REST endpoint

1. Subclass the appropriate Endpoint base in src/sentry/api/endpoints/ (these add Sentry's auth, permissions, rate limiting, and pagination on top of DRF's APIView).
2. Implement get/post/… returning a Response; use serializers from api/serializers/.
3. Register the route in src/sentry/api/urls.py.
4. Add an OpenAPI schema annotation (drf-spectacular) for public endpoints.
5. Add tests under tests/sentry/api/endpoints/.

11.4 Background tasks

Define tasks with the taskworker decorators (namespaced); they execute under sentry run taskworker. Periodic tasks are registered with the scheduler. The older Celery-style tasks are being migrated to taskworker — follow the patterns in src/sentry/taskworker/ and src/sentry/tasks/.

11.5 Migrations

• Generate via sentry django makemigrations (through the django passthrough).
• migrations_lockfile.txt enforces a single migration head; conflicts must be resolved before merge.
• Zero-downtime patterns are enforced by django-pg-zero-downtime-migrations; destructive operations are gated by safe-migration checks.
• Apply with sentry upgrade (production) or sentry django migrate (dev).

11.6 Testing

Type checking: backend via mypy (tools/mypy_helpers/), frontend via pnpm typecheck (tsgo). Linting via eslint, oxfmt, stylelint (frontend) and flake8/ruff-style tooling (backend).

11.7 Feature flags

src/flagpole/ evaluates feature flags from flagpole.yaml definitions (flagpole_eval CLI). Use sentry createflag to scaffold a flag; features gate both backend behavior and frontend surfaces.

12. Troubleshooting (Operations)

Error: SENTRY_SECRET_KEY is undefined

Cause: No secret key configured. Solution: sentry config generate-secret-key, then set SENTRY_SECRET_KEY (env) or system.secret-key (config.yml). Must be ≥ 32 chars. Prevention: Generate and store it in your secrets manager during initial setup.

Error: REDIS_PORT_6379_TCP_ADDR (or SENTRY_REDIS_HOST) is undefined

Cause: Redis isn't linked/configured; the self-hosted config raises this hard. Solution: Set SENTRY_REDIS_HOST (and port/password/db) or link a redis container. Prevention: Treat Redis as a hard dependency in your topology.

Issue: search, dashboards, and stats are empty but issues still appear

Cause: Snuba/ClickHouse or the snuba-writer consumers are down; PostgreSQL-only data still flows. Solution: Check docker compose ps/logs for snuba-* and the relevant sentry run consumer processes; restart and let them catch up on lag. Prevention: Alert on consumer lag and Snuba health.

Issue: events accepted by Relay but never grouped

Cause: Ingest consumers or post-processing tasks aren't running. Solution: Verify sentry run consumer ingest-events (and friends) and sentry run taskworker are healthy; inspect DLQ topics for rejected messages. Prevention: Run consumers and workers under a supervisor with health checks (--healthcheck-file-path).

Issue: migrations fail on upgrade

Cause: Skipped versions, an interrupted previous upgrade, or migration-head conflicts. Solution: Re-run sentry upgrade (idempotent); check migrations_lockfile.txt state; review the failing migration in src/sentry/migrations/. Prevention: Upgrade sequentially through supported versions; never skip the sentry upgrade step.

Issue: emails are never delivered

Cause: The mail backend defaults to dummy until SMTP is configured. Solution: Set SENTRY_EMAIL_HOST (and credentials/TLS), then test with sentry sendmail <file>. Prevention: Configure SMTP during install and verify with a test message.

Issue: native/mobile stack traces are not symbolicated

Cause: Symbolicator isn't running, or debug files weren't uploaded. Solution: Confirm the Symbolicator service is up and that dSYM/ProGuard/debug files are uploaded for the matching release. Prevention: Automate debug-file upload in the build pipeline.

13. Observations & Potential Improvements

Surfaced from a source-level read; these are observations for operators and contributors, not defects in your deployment.

• Operational complexity is the dominant cost. A correct install requires PostgreSQL, Redis, Kafka, ClickHouse/Snuba, Relay, and Symbolicator running in concert. The hard dependency on Redis and the silent dummy mail backend are common first-boot stumbling blocks — front-load these in any internal runbook.
• The analytics plane is a separate failure domain. Because issues persist in PostgreSQL while search/stats live in Snuba, partial outages are easy to misdiagnose. Dashboards that surface Snuba and consumer-lag health pay for themselves quickly.
• Two alerting systems coexist. The legacy rule-based alerts and the newer workflow_engine (detectors/conditions/actions) overlap. When documenting internal alerting standards, pin which surface your org standardizes on.
• Celery→taskworker migration is in flight. Some background work still uses legacy task patterns while new code targets taskworker/taskbroker. Contributors should follow taskworker/ for new tasks and avoid adding Celery-style tasks.
• Pydantic v1 pin. The pydantic<2 constraint will eventually need a major migration; new code should not lean on v1-only behavior in ways that complicate it.
• Frontend state is multi-paradigm. Reflux, MobX, and TanStack Query coexist; new views should prefer TanStack Query for server state to ease the eventual retirement of Reflux.

14. Quick Reference

Service start commands

sentry run web                       # web/API (Granian, :9000)
sentry run taskworker --concurrency 4
sentry run taskworker-scheduler
sentry run consumer ingest-events
sentry upgrade --create-kafka-topics # migrations + bootstrap

Common admin commands

sentry createuser --email [email removed] --superuser
sentry createorg --name Acme --owner-email [email removed]
sentry config generate-secret-key
sentry config set mail.host smtp.acme.com
sentry cleanup --days 90 --concurrency 4
sentry export global backup.json --encrypt-with key.pub

Key paths & ports

Required services checklist

• [ ] PostgreSQL - [ ] Redis - [ ] Kafka - [ ] ClickHouse + Snuba
• [ ] Relay - [ ] Symbolicator - [ ] (optional) Memcached, SMTP, S3/GCS