Jesse A. Green / notebook

Apache Superset — Administrator & Developer Guide

Generated May 2026 from apache/superset at 970b5bc

AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it.

1. Architecture Overview

Apache Superset is an open-source business intelligence platform built on a Python/Flask backend with a React/TypeScript frontend. It connects to external databases via SQLAlchemy, executes queries on behalf of users, and renders interactive visualizations and dashboards in the browser.

The system follows a multi-tier architecture: a Flask web application serves the UI and REST API, Celery workers handle asynchronous tasks (reports, thumbnails, long-running queries), Redis provides caching and message brokering, and a relational database (PostgreSQL or MySQL) stores all metadata.

Architecture Diagram

                        +------------------+
                        |   Web Browser    |
                        +--------+---------+
                                 |
                                 | HTTPS
                                 v
                     +-----------+-----------+
                     |  Reverse Proxy (nginx |
                     |  / Traefik / Ingress) |
                     +-----------+-----------+
                                 |
              +------------------+------------------+
              |                  |                  |
              v                  v                  v
    +---------+------+  +--------+-------+  +-------+--------+
    |  Superset App  |  | Celery Worker  |  | Celery Beat    |
    |  (Flask/       |  | (async tasks,  |  | (scheduler,    |
    |   Gunicorn)    |  |  reports,      |  |  cron-based    |
    |  :8088         |  |  thumbnails)   |  |  triggers)     |
    +--------+-------+  +--------+-------+  +-------+--------+
             |                   |                  |
    +--------+---------+---------+---------+--------+
    |                  |                   |
    v                  v                   v
+---+---+       +------+------+    +------+------+
| Redis |       | PostgreSQL  |    | External    |
| :6379 |       | (metadata)  |    | Databases   |
+-------+       | :5432       |    | (data       |
                +-------------+    |  sources)   |
                                   +-------------+

Optional services:

+------------------+     +------------------+
| WebSocket Server |     |   MCP Service    |
| (Node.js, Redis  |     | (FastMCP, stdio  |
|  streams)        |     |  or HTTP, :5008) |
+------------------+     +------------------+

Component Inventory

Component Technology Path Role
Web Application Flask + Gunicorn superset/ REST API, UI serving, authentication
Frontend SPA React + TypeScript superset-frontend/ Dashboard, chart, SQL Lab UI
Celery Worker Celery + Redis superset/tasks/ Async query execution, reports, thumbnails
Celery Beat Celery Beat superset/tasks/scheduler.py Periodic task scheduling
Metadata Database PostgreSQL / MySQL External User data, dashboard/chart definitions, permissions
Cache / Broker Redis External Result caching, Celery message broker
WebSocket Service Node.js superset-websocket/ Real-time async event streaming
MCP Service FastMCP (Python) superset/mcp_service/ Model Context Protocol integration
Extensions CLI Click (Python) superset-extensions-cli/ Build, validate, and bundle Superset extensions
Embedded SDK TypeScript superset-embedded-sdk/ Embed dashboards in external applications
Documentation Docusaurus 3 docs/ Project documentation site
Helm Chart Kubernetes helm/superset/ Kubernetes deployment

Technology Stack

Layer Technologies
Backend Python 3.9+, Flask, Flask-AppBuilder, SQLAlchemy, Alembic, Celery, Gunicorn
Frontend TypeScript, React 17, Redux, Ant Design 5, Emotion, Webpack 5, ECharts 5, D3.js, deck.gl 9
Database Engines 73 engine specs via SQLAlchemy dialects
Testing pytest (backend), Jest 30 (frontend), Cypress, Playwright
Build setuptools (backend), npm/yarn + webpack (frontend)
Containerization Docker, Docker Compose, Helm

Data Flow

Chart rendering:

  1. User configures chart in Explore UI
  2. Frontend sends form data to /api/v1/chart/data/ or the Explore API
  3. Backend resolves dataset, builds SQL query from form data
  4. SQLAlchemy executes query against the external database
  5. Results are post-processed (aggregation, time comparison, percentage calculations)
  6. JSON results returned to frontend for ECharts/D3/table rendering
  7. Results optionally cached in Redis with a generated cache key

Report execution:

  1. Celery Beat triggers reports.scheduler every minute
  2. Scheduler identifies due reports based on cron schedule and timezone
  3. Worker executes the report: renders dashboard/chart screenshot or runs SQL query
  4. Output (PNG/PDF/CSV) delivered via email (SMTP) or Slack webhook
  5. Execution logged; stale logs pruned daily at 00:10

Note

The SupersetAppInitializer class in superset/initialization/__init__.py orchestrates all component registration: 30+ REST API views, web views, menu items, and Celery configuration are wired together during app startup via init_views().


2. System Requirements

Hardware

Deployment CPU Memory Disk
Development 2 cores 4 GB 10 GB
Small production (<50 users) 4 cores 8 GB 40 GB
Medium production (<500 users) 8 cores 16 GB 100 GB
Large production (500+ users) 16+ cores 32+ GB 200+ GB

Software Dependencies

Component Version Purpose
Python 3.9+ Backend runtime
Node.js 18+ Frontend build
npm 9.0.0+ Package management
PostgreSQL 12+ Metadata database
Redis 6+ Cache and message broker
Docker 20+ Containerization
Kubernetes 1.24+ Orchestration (Helm deployment)

Key Python Dependencies

Package Purpose
Flask Web framework
Flask-AppBuilder Authentication, RBAC, UI scaffolding
SQLAlchemy Database ORM and connection management
Alembic Database migrations
Celery Async task queue
Gunicorn Production WSGI server

Key Frontend Dependencies

Package Version Purpose
React ^17.0.2 UI framework
Ant Design ^5.26.0 Component library
ECharts ^5.6.0 Charting engine
deck.gl ~9.2.5 Geospatial visualizations
Redux ^4.2.1 State management
Emotion ^11.14.0 CSS-in-JS styling
AG Grid 35.2.1 Advanced data tables
Mapbox GL ^3.22.0 Map rendering
D3.js Various Data visualization primitives

3. Configuration Reference

Configuration Files

File Format Location Purpose
superset_config.py Python PYTHONPATH Primary configuration
superset_config_docker.py Python /app/pythonpath/ Docker production overrides
superset_config_docker_light.py Python /app/pythonpath/ Lightweight mode (no Redis/Celery)
superset_test_config_light.py Python /app/pythonpath/ Testing configuration
values.yaml YAML helm/superset/ Helm chart values

Environment Variables

Variable Type Default Description Required?
SUPERSET_SECRET_KEY string (none) Cryptographic key for sessions and CSRF Yes (production)
SUPERSET_CONFIG string superset.config Python module path for custom config No
SUPERSET_APP_ROOT string / URL subpath for non-root deployment No
SUPERSET_ENV string (none) Environment identifier No
SUPERSET_LOG_LEVEL string INFO Log level (DEBUG, INFO, WARNING, ERROR) No
SUPERSET_LOAD_EXAMPLES string (none) Load example data on init (yes/no) No
SUPERSET_TESTENV string false Enable test environment mode No
FLASK_DEBUG string (none) Enable Flask debug mode No
MAPBOX_API_KEY string (none) API key for Mapbox visualizations No
TALISMAN_ENABLED string True Enable Flask-Talisman security headers No
DATABASE_DIALECT string (none) Metadata DB dialect (e.g., postgresql) Docker only
DATABASE_USER string (none) Metadata DB username Docker only
DATABASE_PASSWORD string (none) Metadata DB password Docker only
DATABASE_HOST string (none) Metadata DB host Docker only
DATABASE_PORT string (none) Metadata DB port Docker only
DATABASE_DB string (none) Metadata DB name Docker only
REDIS_HOST string redis Redis host No
REDIS_PORT string 6379 Redis port No
REDIS_CELERY_DB string 0 Redis DB index for Celery broker No
REDIS_RESULTS_DB string 1 Redis DB index for Celery results No
REDIS_CACHE_DB string 4 Redis DB index for caching (test config) No
ADMIN_PASSWORD string (none) Initial admin user password Docker only
CYPRESS_CONFIG string (none) Enable Cypress test mode Dev only
SUPERSET_TESTENV string false Enable test environment behavior Dev only
DEV_MODE string (none) Enable development mode Dev only
GITHUB_SHA string (none) Git commit SHA for version display CI only
GITHUB_HEAD_REF string (none) Git branch name CI only
GITHUB_REF_NAME string (none) Git ref name CI only
GITHUB_TOKEN string (none) GitHub access token for changelog CI only

Important

SUPERSET_SECRET_KEY is the most critical configuration value. If not set, Superset will refuse to start in non-debug environments. Generate a strong key with openssl rand -base64 42 and store it securely. Changing this key invalidates all existing sessions and encrypted database passwords — run superset re_encrypt_secrets after changing it.

Frontend Environment Variables

Variable Default Description
NODE_ENV (none) Node environment (development/production)
WEBPACK_MODE (none) Webpack build mode
WEBPACK_DEVSERVER_PORT (none) Development server port
WEBPACK_DEVSERVER_HOST (none) Development server host
BUNDLE_ANALYZER (none) Enable webpack bundle analyzer
DISABLE_TS_CHECKER (none) Skip TypeScript type checking in builds
SCARF_ANALYTICS (none) Scarf analytics toggle
REDUX_DEFAULT_MIDDLEWARE (none) Redux middleware configuration
PLAYWRIGHT_BASE_URL (none) Base URL for Playwright tests
PLAYWRIGHT_ADMIN_USERNAME (none) Admin username for Playwright tests
PLAYWRIGHT_ADMIN_PASSWORD (none) Admin password for Playwright tests

MCP Service Environment Variables

Variable Default Description
FASTMCP_TRANSPORT stdio Transport mode: stdio or http
FASTMCP_HOST 127.0.0.1 MCP service bind host
FASTMCP_PORT 5008 MCP service bind port
SQLALCHEMY_DEBUG (none) Enable SQLAlchemy debug logging in MCP

Caching Configuration

The Docker development config sets up Redis-backed caching:

CACHE_CONFIG = {
    "CACHE_TYPE": "RedisCache",
    "CACHE_DEFAULT_TIMEOUT": 300,
    "CACHE_KEY_PREFIX": "superset_",
    "CACHE_REDIS_HOST": REDIS_HOST,
    "CACHE_REDIS_PORT": REDIS_PORT,
    "CACHE_REDIS_DB": REDIS_RESULTS_DB,
}

Configure separate caches for different subsystems:

  • CACHE_CONFIG — default cache
  • DATA_CACHE_CONFIG — chart data results
  • FILTER_STATE_CACHE_CONFIG — native filter state
  • EXPLORE_FORM_DATA_CACHE_CONFIG — explore form state

Celery Configuration

class CeleryConfig:
    broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
    result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
    worker_prefetch_multiplier = 1
    task_acks_late = False
    imports = (
        "superset.sql_lab",
        "superset.tasks.scheduler",
        "superset.tasks.thumbnails",
        "superset.tasks.cache",
    )
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=10, hour=0),
        },
    }

CELERY_CONFIG = CeleryConfig

Tip

For production, set worker_prefetch_multiplier = 1 and task_acks_late = True to prevent task loss if a worker crashes mid-execution. Adjust concurrency via CELERYD_CONCURRENCY (default: 2 for dev).

Feature Flags Reference

Flag Default Description
ALERT_REPORTS False Enable scheduled reports and alerts
TAGGING_SYSTEM False Enable the tag management system
THUMBNAILS False Enable chart/dashboard screenshot caching
DYNAMIC_PLUGINS False Enable the dynamic plugin manager
CSS_TEMPLATES False Enable CSS template customization
ENABLE_EXTENSIONS False Enable the extension system
ENABLE_SUPERSET_META_DB False Allow Superset as its own data source
GLOBAL_TASK_FRAMEWORK False Enable the task management system
DATASET_FOLDERS False Enable dataset folder organization
ALERT_REPORTS_NOTIFICATION_DRY_RUN False Dry-run mode for report notifications

Key Application Settings

Setting Default Description
ROW_LIMIT 50000 Default query row limit
SAMPLES_ROW_LIMIT 1000 Data preview row limit
SQLLAB_CTAS_NO_LIMIT True Remove row limit for CTAS operations
DEFAULT_VIZ_TYPE "table" Default chart visualization type
HASH_ALGORITHM "sha256" Hashing algorithm (FedRAMP compliance)
AUTH_USER_REGISTRATION varies Enable user self-registration
SQLALCHEMY_TRACK_MODIFICATIONS False SQLAlchemy event tracking (keep False)

4. Deployment & Scaling

Docker Compose Deployment

The standard Docker Compose deployment includes:

Container Role Default Port
superset_app Web application (Gunicorn) 8088
superset_worker Celery worker —
superset_beat Celery beat scheduler —
superset_db PostgreSQL metadata database 5432
superset_redis Redis cache and broker 6379

Docker entrypoint boot modes:

Mode Command Description
app flask run -p $PORT --reload --debugger Development server
app-gunicorn Gunicorn with configurable workers Production server
worker celery --app=superset.tasks.celery_app:app worker -O fair Celery worker
beat celery --app=superset.tasks.celery_app:app beat Celery scheduler
mcp superset mcp run --host 0.0.0.0 --port $MCP_PORT MCP service

Initialization scripts:

  • docker-init.sh — database migration, admin user creation, role setup, optional example loading
  • docker-bootstrap.sh — dependency installation and configuration bootstrap
  • run-server.sh — Gunicorn production startup

Gunicorn Configuration

Variable Default Description
SUPERSET_BIND_ADDRESS 0.0.0.0 Bind address
SUPERSET_PORT 8088 Application port
SERVER_WORKER_AMOUNT 1 Number of Gunicorn workers
SERVER_WORKER_CLASS gthread Worker class (gthread, gevent, sync)
SERVER_THREADS_AMOUNT 20 Threads per worker
GUNICORN_LOGLEVEL info Log verbosity
GUNICORN_TIMEOUT 60 Worker timeout (seconds)
GUNICORN_KEEPALIVE 2 Keep-alive timeout
WORKER_MAX_REQUESTS 0 Max requests before worker restart (0=disabled)
WORKER_MAX_REQUESTS_JITTER 0 Jitter range for max requests
SERVER_LIMIT_REQUEST_LINE 0 Max request line size (0=unlimited)
SERVER_LIMIT_REQUEST_FIELD_SIZE 0 Max header field size (0=unlimited)

Tip

For production, set SERVER_WORKER_AMOUNT to 2 * CPU_cores + 1. Use gthread worker class with 20 threads for best throughput with database-bound workloads. Set GUNICORN_TIMEOUT to at least 120 for complex dashboard loads.

Kubernetes (Helm) Deployment

The Helm chart (v0.15.4, appVersion 5.0.0) deploys Superset with:

Chart dependencies:

  • PostgreSQL 16.7.27 (Bitnami, OCI registry)
  • Redis 17.9.4 (Bitnami, OCI registry)

Key Helm values:

# Image
image:
  repository: apachesuperset.docker.scarf.sh/apache/superset
  pullPolicy: IfNotPresent

# Service
service:
  type: ClusterIP
  port: 8088

# Ingress
ingress:
  enabled: false
  # annotations, hosts, tls configuration

# SupersetNode (main application)
supersetNode:
  replicas:
    enabled: true
    replicaCount: 1
  autoscaling:
    minReplicas: 1
    maxReplicas: 100
    targetCPUUtilizationPercentage: 80

# Database connections
supersetNode:
  connections:
    db_type: postgresql
    db_host: "{{ .Release.Name }}-postgresql"
    db_port: "5432"
    db_user: superset
    db_pass: superset
    db_name: superset
    redis_host: "{{ .Release.Name }}-redis-headless"
    redis_port: "6379"

Health probes:

Probe Path Initial Delay Failure Threshold
Startup /health 15s 60
Liveness /health 15s 3
Readiness /health 15s 3

Configuration injection:

Method Purpose
configFromSecret Mount superset_config.py from a Kubernetes Secret
envFromSecret Inject env vars from a Secret
envFromSecrets Inject from multiple Secrets
extraEnv Additional environment variables
extraConfigs ConfigMap files (YAML, JSON, etc.)
configOverrides Python config file overrides
bootstrapScript Custom bootstrap shell script
configMountPath /app/pythonpath
extraConfigMountPath /app/configs

Init containers:

  • wait-for-postgres — uses dockerize to wait for database readiness (timeout: 120s)
  • Workers additionally wait for Redis before starting

Warning

The default Helm values set runAsUser: 0 (root). For production Kubernetes deployments, change this to a non-root user (1000+) and ensure volume permissions match.

Scaling Guidelines

Component Scaling Strategy
Web application Horizontal: increase replicaCount or HPA
Celery workers Horizontal: add worker replicas, adjust concurrency
Redis Vertical: increase memory; consider Redis Cluster for large deployments
PostgreSQL Vertical: increase resources; consider read replicas
WebSocket service Horizontal: load-balance across instances

5. Operations

Health Monitoring

Health check endpoints:

Endpoint Response Purpose
GET /health OK Application liveness
GET /healthcheck OK Alias for /health
GET /ping OK Alias for /health
GET /version JSON with version info Version and build information

Note

The health endpoint at http://localhost:${SUPERSET_PORT}/${SUPERSET_APP_ROOT}/health is the standard target for Docker, Kubernetes, and load balancer health checks.

Celery Task Management

Monitoring tasks:

The Task REST API (/api/v1/tasks/) provides:

  • GET / — list tasks with filters (type, status, scope)
  • GET /<uuid> — retrieve task details
  • DELETE /<uuid> — cancel a running task
  • GET /<uuid>/status — check task status

Common Celery tasks:

Task Schedule Purpose
reports.scheduler Every minute Trigger due reports and alerts
reports.prune_log Daily at 00:10 Clean up old report execution logs

Thumbnail Management

Compute thumbnails for all dashboards and charts:

superset compute_thumbnails
Flag Description
--asynchronous / -a Run as Celery tasks instead of synchronously
--dashboards_only / -d Only process dashboards
--charts_only / -c Only process charts
--force / -f Force refresh even if cached
--model_id / -i Process a specific dashboard/chart by ID

Database Operations

Update a database URI:

superset set_database_uri -d "My Database" -u "postgresql+psycopg2://new_user:pass@host/db"

Use --skip_create / -s to prevent auto-creating the database if it does not exist.

Re-encrypt secrets after key rotation:

superset re_encrypt_secrets --previous_secret_key "old_key"

Caution

If SUPERSET_SECRET_KEY is changed without running re_encrypt_secrets, all stored database passwords become unreadable. Always back up the metadata database before key rotation.

Import/Export Operations

Full asset export (API):

curl -X GET "http://localhost:8088/api/v1/assets/export/" \
  -H "Authorization: Bearer $TOKEN" \
  -o superset_assets_$(date +%Y%m%d).zip

Full asset import (API):

curl -X POST "http://localhost:8088/api/v1/assets/import/" \
  -H "Authorization: Bearer $TOKEN" \
  -F "bundle=@superset_assets.zip" \
  -F "passwords={\"databases/My_DB.yaml\": \"db_password\"}" \
  -F "ssh_tunnel_passwords={}" \
  -F "ssh_tunnel_private_keys={}" \
  -F "ssh_tunnel_private_key_passwords={}" \
  -F "encrypted_extra_secrets={}"

CLI export/import:

Command Description
superset export_dashboards -f file.zip Export all dashboards
superset import_dashboards -p file.zip -u admin Import dashboards
superset export_datasources -f file.zip Export all datasources
superset import_datasources -p file.zip Import datasources
superset import_directory <dir> --overwrite Import from directory

Legacy format commands (for older Superset versions):

Command Description
superset legacy_export_dashboards -f file.json Export as JSON
superset legacy_import_dashboards -p file.json Import from JSON
superset legacy_export_datasources -f file.yaml Export as YAML
superset legacy_import_datasources -p file.yaml Import from YAML
superset legacy_export_datasource_schema Export datasource schema

Note

The modern ZIP-based format supports sparse updates (--sparse flag) and handles SSH tunnel credentials, encrypted extras, and database passwords during import.

Backup Strategy

  • [x] Back up PostgreSQL metadata database daily
  • [x] Export assets via API or CLI before major upgrades
  • [x] Store SUPERSET_SECRET_KEY securely and separately
  • [ ] Test restore procedure quarterly
  • [ ] Monitor disk usage for Redis persistence

6. Performance

Caching Strategy

Superset uses a multi-layer caching architecture:

Cache Purpose Recommended Backend
Default cache General-purpose Redis
Data cache Chart query results Redis
Filter state cache Native filter state Redis
Explore form data cache Explore view state Redis
Thumbnail cache Dashboard/chart screenshots Redis or filesystem

Cache key generation: Superset generates cache keys from a dictionary of query parameters using generate_cache_key(). Cache entries track the datasource for invalidation purposes.

ETag caching: The @etag_cache decorator provides HTTP-level caching with Last-Modified, Expires, and conditional request (If-None-Match) support for API responses.

Cache warming: Pre-warm chart caches using:

POST /api/v1/chart/<id>/warm_up_cache

Query Optimization

  • Set ROW_LIMIT appropriately (default: 50,000) — lower limits reduce response time
  • Use SAMPLES_ROW_LIMIT (default: 1,000) for data previews
  • Enable query cost estimation for databases that support it
  • Configure SQLLAB_CTAS_NO_LIMIT = True to avoid row limits on CTAS operations
  • Use SQLLAB_TIMEOUT to prevent runaway queries

Frontend Performance

  • Enable NODE_OPTIONS="--max-old-space-size=8192" for builds
  • The frontend uses lazy-loaded React components for code splitting
  • Webpack bundle analyzer available via BUNDLE_ANALYZER=true
  • TypeScript checking can be skipped during development with DISABLE_TS_CHECKER=true

Tip

For dashboards with many charts, use the dashboard tab feature to split content across tabs. Only visible tabs execute queries, reducing initial load time.


7. Security

Authentication

Superset uses Flask-AppBuilder for authentication, supporting:

  • Database authentication — username/password stored in the metadata DB
  • OAuth/OpenID — Google OAuth, Okta, Auth0, etc.
  • LDAP — Active Directory / LDAP integration
  • Remote user — trust headers from a reverse proxy

Configure via AUTH_TYPE and related settings in superset_config.py.

User registration: Controlled by AUTH_USER_REGISTRATION. When enabled, new users can self-register with email activation via SupersetRegisterUserView.

Authorization (RBAC)

Superset implements role-based access control through Flask-AppBuilder:

Built-in roles:

Role Permissions
Admin Full access to all features and data
Alpha Access to all data sources, can create charts/dashboards
Gamma Access only to data sources explicitly granted
Public Unauthenticated access (if enabled)
sql_lab Access to SQL Lab features

Custom roles can be created with fine-grained permissions on specific data sources, views, and actions.

Row-Level Security (RLS)

RLS rules add automatic WHERE clauses to queries based on user roles:

  • Rules map roles to tables with SQL filter clauses
  • Multiple filter types supported for different security models
  • Group keys organize rules logically
  • Rules compose — multiple rules on the same table are combined

Security REST API:

Endpoint Method Description
/api/v1/security/csrf_token/ GET Generate CSRF token
/api/v1/security/guest_token/ POST Create guest token for embedded dashboards
/api/v1/security/roles/search/ GET List roles with user and permission IDs

Guest Tokens (Embedded Security)

Guest tokens enable secure embedded dashboard access:

# Token request payload
{
    "user": {"username": "guest", "first_name": "Guest"},
    "resources": [{"type": "dashboard", "id": "<dashboard-uuid>"}],
    "rls": [{"clause": "team_id = 'sales'"}]
}

The embedded SDK automatically refreshes tokens before expiry using JWT-based timing calculations.

Database Connection Security

URI validation: The check_sqlalchemy_uri() function enforces a blocklist:

  • Blocked: SQLite, Shillelagh (local file access risk)
  • Administrators can customize the blocklist via configuration

Encrypted credentials: Database passwords are stored encrypted using SUPERSET_SECRET_KEY. SSH tunnel credentials (passwords, private keys) are similarly encrypted.

Warning

Never expose SUPERSET_SECRET_KEY in logs, version control, or error messages. If compromised, rotate the key immediately and run superset re_encrypt_secrets to re-encrypt all stored credentials.

Security Headers

Flask-Talisman provides security headers (CSP, HSTS, etc.). Controlled via TALISMAN_ENABLED (default: True).

Distributed Locking

Superset uses distributed locks (via the key-value store) to prevent concurrent operations on shared resources. Lock entries are stored in the KeyValueEntry table with the lock resource type.


8. Developer Internals

Repository Structure

apache/superset/
+-- superset/                    # Python backend (Flask application)
|   +-- __init__.py              # App factory, extensions
|   +-- app.py                   # create_app() factory function
|   +-- config.py                # Default configuration (2000+ lines)
|   +-- initialization/          # SupersetAppInitializer
|   +-- cli/                     # CLI commands (Click)
|   +-- views/                   # Web views and API endpoints
|   +-- models/                  # SQLAlchemy models
|   +-- daos/                    # Data Access Objects
|   +-- commands/                # Business logic commands
|   +-- security/                # SupersetSecurityManager
|   +-- db_engine_specs/         # 73 database engine specifications
|   +-- charts/                  # Chart REST API and models
|   +-- dashboards/              # Dashboard REST API and models
|   +-- datasets/                # Dataset REST API and models
|   +-- databases/               # Database REST API and models
|   +-- sqllab/                  # SQL Lab API and execution
|   +-- reports/                 # Report scheduling API
|   +-- tags/                    # Tagging system
|   +-- key_value/               # Key-value store
|   +-- embedded/                # Embedded dashboard support
|   +-- mcp_service/             # Model Context Protocol service
|   +-- tasks/                   # Celery tasks and task API
|   +-- utils/                   # Utility functions (45 files)
|   +-- common/                  # Shared enums and utilities
|   +-- extensions/              # Flask extensions and meta DB
|   +-- migrations/              # Alembic database migrations
|   +-- sql/                     # SQL parsing and validation
|   +-- sql_validators/          # SQL validation engines
|   +-- importexport/            # Import/export API
|   +-- row_level_security/      # RLS API
|   +-- themes/                  # Theme definitions
|   +-- translations/            # i18n translation files
|   +-- examples/                # Example datasets and dashboards
|   +-- async_events/            # Async event streaming
|   +-- thumbnails/              # Screenshot generation
|   +-- explore/                 # Explore API
|   +-- datasource/              # Datasource API
|   +-- annotation_layers/       # Annotation API
|   +-- css_templates/           # CSS template API
|   +-- temporary_cache/         # Temporary cache entries
|   +-- advanced_data_type/      # Advanced data type handling
|   +-- distributed_lock/        # Distributed locking
|   +-- cachekeys/               # Cache key tracking
+-- superset-frontend/           # TypeScript/React frontend
|   +-- src/                     # Application source
|   |   +-- components/          # Shared React components
|   |   +-- views/               # Page-level views
|   |   +-- features/            # Feature modules
|   |   +-- explore/             # Chart explore interface
|   |   +-- SqlLab/              # SQL Lab IDE
|   |   +-- embedded/            # Embedded dashboard
|   |   +-- utils/               # Frontend utilities (58 files)
|   |   +-- hooks/               # React hooks (13 files)
|   |   +-- middleware/           # Redux middleware
|   |   +-- core/                # Core commands and utilities
|   |   +-- theme/               # Theme utilities
|   +-- plugins/                 # Visualization plugins
|   |   +-- plugin-chart-echarts/ # ECharts-based charts
|   |   +-- plugin-chart-table/  # Table visualization
|   |   +-- plugin-chart-pivot-table/ # Pivot table
|   |   +-- plugin-chart-ag-grid-table/ # AG Grid table
|   |   +-- preset-chart-deckgl/ # deck.gl geospatial charts
|   |   +-- plugin-chart-word-cloud/ # Word cloud
|   |   +-- plugin-chart-handlebars/ # Handlebars template chart
|   |   +-- plugin-chart-cartodiagram/ # Cartodiagram
|   |   +-- plugin-chart-point-cluster-map/ # Point cluster map
|   |   +-- legacy-plugin-chart-*/ # Legacy chart types
|   |   +-- legacy-preset-chart-nvd3/ # Legacy NVD3 charts
|   +-- packages/                # Shared packages
|   |   +-- superset-core/       # @apache-superset/core
|   |   +-- superset-ui-core/    # @superset-ui/core
|   |   +-- superset-ui-chart-controls/ # Chart control components
|   |   +-- superset-ui-switchboard/ # Cross-frame communication
|   |   +-- generator-superset/  # Yeoman plugin generator
|   +-- playwright/              # E2E tests
|   +-- cypress-base/            # Cypress tests
+-- superset-embedded-sdk/       # Embedded dashboard SDK
+-- superset-extensions-cli/     # Extension management CLI
+-- superset-websocket/          # WebSocket service (Node.js)
+-- superset-core/               # New core package
+-- helm/superset/               # Kubernetes Helm chart
+-- docker/                      # Docker configuration
+-- docs/                        # Docusaurus documentation site
+-- tests/                       # Backend test suite
+-- scripts/                     # Build and utility scripts
+-- RELEASING/                   # Release management tools

REST API Reference

Superset exposes 30+ REST API resource groups registered during initialization. Key APIs:

Data Layer APIs:

API Class Base Path Key Operations
DatabaseRestApi /api/v1/database/ CRUD, test connection, validate SQL, upload files, list schemas/catalogs/tables
DatasetRestApi /api/v1/dataset/ CRUD, column/metric management
DatasourceRestApi /api/v1/datasource/ Unified datasource access

Visualization APIs:

API Class Base Path Key Operations
ChartRestApi /api/v1/chart/ CRUD, screenshots, cache warming, favorites, viz types
ChartDataRestApi /api/v1/chart/data/ Execute chart queries
DashboardRestApi /api/v1/dashboard/ CRUD, screenshots, filters, colors, tabs, embedding, copy
ExploreRestApi /api/v1/explore/ Assemble explore context from form data

Administrative APIs:

API Class Base Path Key Operations
SecurityRestApi /api/v1/security/ CSRF tokens, guest tokens
RLSRestApi /api/v1/rowlevelsecurity/ Row-level security rules CRUD
RoleRestAPI /api/v1/security/roles/ Role management
UserRestApi /api/v1/user/ User management
ReportScheduleRestApi /api/v1/report/ Report/alert scheduling
TagRestApi /api/v1/tag/ Tag management
ImportExportRestApi /api/v1/assets/ Bulk import/export
TaskRestApi /api/v1/tasks/ Task management

Utility APIs:

API Class Base Path Key Operations
SqlLabRestApi /api/v1/sqllab/ SQL execution, formatting, cost estimation
QueryRestApi /api/v1/query/ Query history
SavedQueryRestApi /api/v1/saved_query/ Saved queries
LogRestApi /api/v1/log/ Audit log
CacheRestApi /api/v1/cache/ Cache management
AnnotationRestApi /api/v1/annotation_layer/ Annotation layers
CssTemplateRestApi /api/v1/css_template/ CSS templates
ThemeRestApi /api/v1/theme/ Theme management

Database API extended routes:

Method Path Description
GET /<id>/tables List database tables
GET /<id>/table_metadata Get table schema
GET /<id>/select_star Preview SELECT * query
GET /<id>/catalogs List catalogs (multi-catalog support)
GET /<id>/schemas List schemas
POST /<id>/test_connection Test database connectivity
POST /<id>/validate_parameters Validate configuration
POST /<id>/validate_sql Validate SQL syntax
POST /upload Upload data files (CSV, Excel, Parquet)
POST /<id>/sync_permissions Sync database permissions

Dashboard API extended routes:

Method Path Description
GET /<id_or_slug> Get dashboard by ID or slug
GET /<id>/charts List charts in dashboard
GET /<id>/datasets List datasets used
GET /<id>/tabs Get tab configuration
POST /<id>/favorite Add to favorites
DELETE /<id>/favorite Remove from favorites
GET /<id>/thumbnail Dashboard thumbnail
POST /<id>/screenshot Capture screenshot
PUT /<id>/filters Update native filters
PUT /<id>/chart_customizations Update chart customizations
PUT /<id>/colors Update color configuration
POST /<id>/copy_dash Copy dashboard
POST /embedded Create embedded dashboard config
GET /embedded/<uuid> Get embedded dashboard

CLI Command Reference

The superset CLI is registered as a console script entry point: superset=superset.cli.main:superset.

Core commands:

Command Description
superset init Create default roles and permissions
superset version [-v] Show version (verbose: include build info)
superset db upgrade Run Alembic database migrations
superset fab create-admin Create an admin user
superset run -p 8088 Start Flask development server

Data management:

Command Description
superset load_examples [-t] [-b] [-m] [-f] Load example datasets
superset export_example -d <id> [-n name] [-o dir] Export dashboard as example
superset export_dashboards -f <file> Export all dashboards
superset export_datasources -f <file> Export all datasources
superset import_dashboards -p <file> [-u user] Import dashboards
superset import_datasources -p <file> Import datasources
superset import_directory <dir> [-o] [-f] Import from directory

Administration:

Command Description
superset set_database_uri -d <name> -u <uri> Update database connection URI
superset sync_tags Synchronize tag associations
superset update_api_docs Regenerate API documentation
superset re_encrypt_secrets [-a <old_key>] Re-encrypt after key rotation
superset factory_reset --username <admin> Factory reset (destructive)
superset compute_thumbnails [-a] [-d] [-c] [-f] Generate thumbnails
superset test_db <uri> [-c <connect_args>] Test database connection
superset load_test_users Create test user accounts

Visualization migration:

Command Description
superset migrate_viz upgrade -t <type> [--id <id>] Migrate charts to new viz type
superset migrate_viz downgrade -t <type> [--id <id>] Revert chart migration

MCP service:

Command Description
superset mcp run [--host] [--port] [--debug] Start MCP server

Extensions CLI (separate package: superset-extensions-cli):

Command Description
superset-extensions-cli init Scaffold a new extension
superset-extensions-cli validate Validate extension structure
superset-extensions-cli build Build extension
superset-extensions-cli bundle [-o file] Bundle extension for distribution
superset-extensions-cli dev Start extension in dev mode
superset-extensions-cli update [--version] [--license] Update extension metadata

Module Reference (Key Backend Modules)

superset/app.py — Application factory. create_app() loads configuration from the SUPERSET_CONFIG environment variable (default: superset.config), initializes SupersetAppInitializer, and returns the Flask app. Supports APPLICATION_ROOT for subdirectory deployments.

superset/initialization/__init__.py — SupersetAppInitializer orchestrates startup: registers 30+ API views, configures Celery, sets up menu items, runs pre/post-init hooks.

superset/models/core.py — Core SQLAlchemy models including Database (connection management with encrypted passwords, OAuth2, SSH tunnels, capabilities like allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml), CssTemplate, Theme, AnnotationLayer, Annotation, CacheKey, KeyValue.

superset/db_engine_specs/ — 73 database engine specification modules. Each spec extends BaseEngineSpec with database-specific behavior: time grain expressions, query cost estimation, error message parsing, connection validation, schema browsing, file upload support. Specs are loaded dynamically via load_engine_specs() and can be extended via the superset.db_engine_specs entry point group.

superset/security/manager.py — SupersetSecurityManager extends Flask-AppBuilder's security manager with Superset-specific authorization: database URI validation, guest token management, row-level security enforcement, permission syncing.

superset/sql/parse.py — SQL parsing engine (51K chars). Parses SQL for validation, extracts table references, validates clauses, and handles Jinja template rendering.

superset/daos/ — 19 Data Access Object modules providing database query patterns for all entity types.

superset/commands/ — Business logic layer implementing the command pattern. Commands encapsulate create, update, delete, export, and import operations with validation and authorization.

superset/views/ — 29 view modules including Api (core query execution), SupersetAuthView (login), SupersetRegisterUserView (registration), and health endpoints.

superset/utils/ — 45 utility modules including caching (cache.py), core utilities (core.py), decorators, date parsing, SQL formatting, and email sending.

superset/extensions/ — 15 Flask extension modules including the meta database (metadb.py — registers Superset as a Shillelagh-backed SQLAlchemy dialect at superset://), local extensions watcher, and feature flag manager.

Extension System

The @apache-superset/core package (v0.1.0-rc2) provides the extension API:

Extension contribution types:

Type Registration Purpose
Views views.registerView() Add UI panels at specific locations
Commands commands.registerCommand() Define named actions (menus, shortcuts, code)
Menus menus.registerMenuItem() Attach commands to menu locations
Editors editors.registerEditor() Replace text editors for specific languages

Extension manifest format (JSON):

  • id — composite ID (publisher.name)
  • publisher, name, displayName, version
  • permissions, dependencies
  • frontend.remoteEntry, frontend.moduleFederationName
  • backend.entrypoint

Extensions use Module Federation for frontend loading and standard Python entry points for backend integration.

WebSocket Service Architecture

The WebSocket service (superset-websocket/) is a standalone Node.js server:

  • Transport: WebSocket over HTTP upgrade
  • Auth: JWT-based (extracted from cookies, configurable algorithms)
  • Backend: Redis Streams for event delivery
  • Metrics: StatsD integration for monitoring
  • Lifecycle: Ping/pong keep-alive, garbage collection for stale channels

Configuration: port, logging, Redis connection, JWT settings, socket management (response timeout, ping interval, GC interval).

MCP Service Architecture

The MCP service (superset/mcp_service/) implements the Model Context Protocol:

  • Server: FastMCP with Flask app context integration
  • Transport: Stdio (default) and HTTP modes
  • Middleware stack: Response size guard, logging, error handling, content stripping
  • Tool registration: Automatic tool discovery and registration
  • Security: Suppresses non-essential output in stdio mode

Database Engine Specification System

The superset/db_engine_specs/ directory contains 73 engine specification files implementing BaseEngineSpec. The system:

  1. Discovers specs via pkgutil.iter_modules() at startup
  2. Loads additional specs from the superset.db_engine_specs entry point group
  3. Matches specs to connections using get_engine_spec(backend, driver)
  4. Falls back to backend-only matching if no driver-specific spec exists

Each spec provides: time grain expressions, date truncation SQL, query cost estimation, error message parsing and categorization, connection parameter validation, catalog/schema/table browsing, file upload handlers, and SQL dialect adjustments.

Testing Strategy

Backend testing:

Type Location Framework
Unit tests tests/unit_tests/ pytest
Integration tests tests/integration_tests/ pytest
Common fixtures tests/common/ pytest

Run tests with:

scripts/tests/run.sh
scripts/tests/run.sh --module tests/charts/api_tests.py
scripts/tests/run.sh --module tests/charts/api_tests.py::TestChartApi::test_get_charts --no-init

Frontend testing:

Type Framework Config
Unit tests Jest 30 jest.config.js
Component tests Testing Library @testing-library/react
E2E tests Playwright playwright.config.ts
E2E tests (legacy) Cypress cypress-base/
Visual tests Storybook 8.6 storybook config

Code quality tools:

Tool Config
ESLint (frontend) .eslintrc.js, custom plugins (i18n-strings, theme-colors, icons)
Prettier prettier v3.8.3
TypeScript v5.4.5, strict mode
Black/flake8 (backend) Standard Python formatting

Entity Relationship Diagram

Generate an ERD of the Superset metadata schema:

python scripts/erd/erd.py -o superset_erd.png

Contributing

Development setup:

  1. Clone the repository
  2. Set up Python virtual environment with pip install -e .
  3. Set up frontend with cd superset-frontend && npm ci
  4. Start PostgreSQL and Redis (via Docker or locally)
  5. Initialize: superset db upgrade && superset fab create-admin && superset init
  6. Start backend: superset run -p 8088 --with-threads --reload
  7. Start frontend: cd superset-frontend && npm run dev-server

Documentation development:

cd docs
yarn install
yarn start          # Quick start (smart generation)
yarn start:full     # Full start (regenerate all)

The docs site uses Docusaurus 3 with four independent versioned sections: User Documentation, Admin Documentation, Developer Docs, and Component Playground. Always use the automated versioning scripts:

yarn version:add:docs 1.2.0
yarn version:remove:docs 1.0.0

Warning

Do NOT use native Docusaurus versioning commands (yarn docusaurus docs:version). They will create version files but fail to update versions-config.json, causing versions to not appear in dropdown menus.

Potential Improvements

Based on source analysis:

  • API server decomposition: api_server.py equivalent modules could be split into domain-specific routers for maintainability
  • SQL parsing consolidation: The 51K-char sql/parse.py handles multiple concerns (parsing, validation, Jinja rendering) that could be separated
  • Engine spec metadata linting: The lint_metadata.py tool validates engine spec completeness — running it regularly with --strict would catch missing required fields
  • Extension system maturity: The extension CLI and @apache-superset/core package are at v0.1.0-rc2 — the contribution model (views, commands, menus, editors) is well-designed but still pre-release

9. Integration Points

Embedded Dashboard SDK

The @superset-ui/embedded-sdk provides:

Function Description
embedDashboard() Main embedding function
getGuestTokenRefreshTiming() Calculate JWT-based refresh interval

SDK capabilities:

  • Automatic guest token refresh before expiry
  • Iframe sandbox and security configuration
  • Filter synchronization and data mask observation
  • Permalink URL resolution with custom callbacks
  • Theme configuration (default/dark/system)
  • Chart state and data payload access
  • Dashboard scroll size measurement for responsive sizing

Superset Meta Database

When ENABLE_SUPERSET_META_DB is enabled, Superset registers:

  • SQLAlchemy dialect: superset:// (via superset.extensions.metadb.SupersetAPSWDialect)
  • Shillelagh adapter: superset.extensions.metadb.SupersetShillelaghAdapter

This allows querying Superset's own data through SQL Lab as if it were an external database.

Email Integration

Superset sends emails for:

  • Report delivery (SMTP with optional SSL/TLS)
  • Alert notifications
  • User registration activation

Configure SMTP settings in superset_config.py. Email content supports inline images and attachments. An email mutator hook allows customizing recipients and content programmatically.

Slack Integration

Reports and alerts can deliver to Slack channels when configured with a Slack API token. Notifications include chart screenshots, dashboard PDFs, or CSV data attachments.

Key-Value Store

The KeyValueEntry model provides a generic key-value store used internally for:

Resource Purpose
dashboard_permalink Shared dashboard state URLs
explore_permalink Shared explore state URLs
sqllab_permalink Shared SQL Lab state URLs
metastore_cache Metadata caching
lock Distributed locking
pkce_code_verifier OAuth2 PKCE code verification

Entries support expiry, UUID-based keys, and multiple codecs (JSON, Pickle). Maximum value size: 2^24-1 bytes (16 MB).

Async Events (WebSocket)

The WebSocket service streams events from Redis Streams to connected browser clients:

  • Channel-based message filtering
  • JWT authentication from cookies
  • Automatic keep-alive via ping/pong
  • StatsD metrics for monitoring
  • Garbage collection for stale connections

Database-Specific Integrations

The 73 engine specs in superset/db_engine_specs/ provide deep integration with specific databases:

  • OAuth2 authentication — for databases requiring token-based auth
  • IAM authentication — AWS IAM auth for Redshift, Athena, etc.
  • Multi-catalog support — browsing catalogs, schemas, and tables
  • Query cost estimation — pre-execution cost preview
  • File upload — CSV, Excel, Parquet upload with database-specific handling
  • SSH tunneling — secure connections through SSH bastions

Note

Third-party database engine specs can be registered via the superset.db_engine_specs Python entry point group, allowing custom databases to integrate without modifying Superset core.