Diagrams Admin & Architecture Guide

This guide is for maintainers, contributors, and architects who need to understand how Diagrams works internally: the global-context rendering model, the operator-overloading DSL, the Graphviz integration, and — most importantly — the icon-to-class auto-generation pipeline that produces the library's ~2,400 provider node classes. It also covers how to add a provider or node, the test strategy, security considerations for the CLI, and known limitations.

For end-user installation and diagram syntax, see the companion User Guide.

1. Executive Summary

Diagrams is a pure-Python library that turns Python code into cloud-architecture images. It is a thin, elegant layer over Graphviz: the library's job is to (1) provide an ergonomic Python DSL for declaring nodes and edges, and (2) attach the correct icon image to each node. All actual graph layout and rendering is delegated to Graphviz via the graphviz Python binding.

The codebase has two clearly separated halves:

This separation is the single most important architectural fact about the project. The runtime is small and stable; the provider surface is large and mechanically generated. Understanding the generator (scripts/, config.py, templates/) is essential before touching anything under diagrams/<provider>/.

2. Architecture Overview

                          Author writes diagram.py
                                    │
                                    ▼
          ┌──────────────────────────────────────────────┐
          │  diagrams/__init__.py  (the runtime core)     │
          │                                               │
          │   Diagram  ── context mgr, holds graphviz.Digraph
          │   Cluster  ── context mgr, builds subgraph     │
          │   Node     ── base; >> << - operator overloads │
          │   Edge     ── connection attrs (label/color)   │
          │                                               │
          │   contextvars: __diagram, __cluster (globals)  │
          └───────────────────┬──────────────────────────┘
                              │ provider node classes subclass Node
                              ▼
          ┌──────────────────────────────────────────────┐
          │ diagrams/<provider>/<category>.py             │
          │   class EC2(_Compute):  _icon = "ec2.png"     │  ← AUTO-GENERATED
          │   _icon_dir = "resources/aws/compute"          │
          └───────────────────┬──────────────────────────┘
                              │ Node._load_icon() resolves PNG path
                              ▼
          ┌──────────────────────────────────────────────┐
          │  graphviz.Digraph  →  dot (native Graphviz)    │
          │  .render()  →  web_service.png / .svg / .pdf   │
          └──────────────────────────────────────────────┘

  Build-time (maintainers only):
     resources/<provider>/*.png  ──►  scripts/generate.py  ──►  diagrams/<provider>/*.py
                          (config.py rules + templates/module.tmpl, via autogen.sh)

The two execution contexts

1. Author runtime — when a user runs python diagram.py. Only diagrams/__init__.py and the provider class files are involved. Graphviz renders the output.
2. Maintainer build — when icons change. autogen.sh preprocesses SVG/PNG resources and regenerates every provider module. End users never run this.

3. The Runtime Core (diagrams/__init__.py)

The entire user-facing runtime is ~620 lines in a single module. Four classes and a global context.

3.1 Global context via contextvars

Diagrams avoids threading a diagram object through every call by storing the "current diagram" and "current cluster" in module-level contextvars.ContextVar objects:

__diagram = contextvars.ContextVar("diagrams")
__cluster = contextvars.ContextVar("cluster")

def getdiagram(): ...   # returns current Diagram or None
def setdiagram(d): ...
def getcluster(): ...   # returns current Cluster or None
def setcluster(c): ...

• Diagram.__enter__ calls setdiagram(self); __exit__ calls render(), removes the intermediate .dot file, then setdiagram(None).
• Cluster.__enter__ calls setcluster(self); __exit__ attaches its subgraph to its parent (or the diagram) and restores the previous cluster.
• Node.__init__ reads getdiagram() and getcluster() to know where to attach itself.

Note

Using contextvars (rather than plain globals) means the context is coroutine/thread-local-aware. However, the library is not designed for concurrent diagram construction in the same context — build one diagram at a time within a with block.

Important

If getdiagram() returns None when a Node or Cluster is constructed, the constructor raises EnvironmentError("Global diagrams context not set up"). This is the enforcement mechanism that requires all nodes to live inside a with Diagram(...) block.

3.2 Diagram

Diagram wraps a graphviz.Digraph. Construction:

1. Validates theme against THEMES (raises ValueError if unknown) and stores the theme's color config.
2. Derives filename from name if not given ("Web Service" → web_service); falls back to diagrams_image if both are empty.
3. Creates graphviz.Digraph(name, filename=filename, strict=strict).
4. Applies _default_graph_attrs, _default_node_attrs, _default_edge_attrs (the visual defaults), then the theme edge color.
5. Validates and applies direction → rankdir, curvestyle → splines.
6. Validates outformat (string or list) against the allowed set.
7. Merges user-supplied graph_attr / node_attr / edge_attr over the defaults.

Validation tables (class attributes):

Default visual attributes (the source of the library's signature look):

_default_graph_attrs = {"pad": "2.0", "splines": "spline", "nodesep": "0.70",
                        "ranksep": "0.90", "fontname": "Sans-Serif",
                        "fontsize": "15", "fontcolor": "#2D3436"}
_default_node_attrs  = {"shape": "box", "style": "rounded", "fixedsize": "true",
                        "width": "1.4", "height": "1.4", "labelloc": "b",
                        "imagepos": "tc", "imagescale": "true", "fontname": "Sans-Serif",
                        "fontsize": "13", "fontcolor": "#2D3436"}
_default_edge_attrs  = {"color": "#495057", "arrowsize": "0.8"}

render() calls self.dot.render(format=..., view=self.show, quiet=True) once per requested format (it loops if outformat is a list).

3.3 Themes

Themes are a recent feature. THEMES is a dict of five palettes; each defines a tuple of cluster background colors (indexed by nesting depth), a pencolor (cluster border), and an edgecolor:

Cluster reads the active diagram's _theme_config to pick its background color: coloridx = self.depth % len(bgcolors), so deeply nested clusters cycle through the palette.

Note

Because theme/autolabel post-date the 0.25.1 PyPI release, the version string in pyproject.toml lags the source feature set (tracked upstream as a version-mismatch issue). Documentation generated from source — like this guide — reflects the code at commit 23f06b4, which includes themes.

3.4 Node — the base of every provider class

Node is the abstract base. Every provider class is ultimately class X(Node). Key class attributes a subclass overrides: _provider, _type, _icon_dir, _icon, _height.

Construction logic:

• Generates a random uuid4().hex node id unless nodeid= is passed.
• Requires an active diagram (else EnvironmentError).
• If the diagram has autolabel=True, prepends the class name to the label.
• If the class has an _icon, builds Graphviz attrs {"shape": "none", "height": ..., "image": self._load_icon()}; otherwise an empty attrs dict (a labelled box).
• _load_icon() resolves the icon path relative to the package directory: os.path.join(<package parent>, self._icon_dir, self._icon).
• Attaches itself to the current cluster if one is active, else to the diagram.

Tip

The Custom node overrides _load_icon() to return a caller-supplied path verbatim, which is how user images bypass the resources/ lookup.

3.5 The connection DSL — operator overloading

The >>, <<, and - syntax is implemented by overloading Python operators on Node and Edge:

The "r"-prefixed reflected dunders (__rrshift__, etc.) exist specifically so that a Python list on the left of an operator works — e.g. [a, b] >> c. Lists don't define these operators, so Python falls back to the right operand's reflected method, which iterates the list and connects each element. This is why you can fan-in/fan-out with list literals but cannot put a list on both sides.

Each operator returns the right-hand operand so chains (a >> b >> c) compose left-to-right. The actual edge is added on the diagram (not the cluster) via Diagram.connect() → self.dot.edge(...), because Graphviz requires cross-cluster edges to be declared at the top level.

3.6 Edge

Edge holds connection attributes. The constructor accepts node, forward, reverse, label, color, style, and arbitrary **attrs (passed straight to Graphviz). The computed attrs property maps the forward/reverse booleans to Graphviz's dir attribute (forward/back/both/none). Edge overloads the same operators as Node to support edge chaining and labeling mid-flow.

Note

A historical workaround lives here: edge labels are stored under the Graphviz label key (an earlier xlabel approach caused misaligned labels — see upstream issue #83).

3.7 Cluster / Group

Cluster creates a Graphviz subgraph whose name is prefixed cluster_ (Graphviz treats cluster*-named subgraphs specially, drawing a bounding box). Depth is computed from the parent cluster; the background color is chosen from the theme palette by depth. Group = Cluster is a simple alias.

Warning

A FIXME in the source notes that per-cluster direction does not currently work — Graphviz cannot reliably render a subgraph with a different rank direction than its parent. Don't rely on Cluster(direction=...).

4. The CLI (diagrams/cli.py)

The console entry point is intentionally minimal:

def run() -> int:
    parser = argparse.ArgumentParser(description="Run diagrams code files in a diagrams environment.")
    parser.add_argument("paths", metavar="path", type=str, nargs="+",
                        help="a Python file containing diagrams code")
    args = parser.parse_args()
    for path in args.paths:
        with open(path, encoding='utf-8') as f:
            exec(f.read())
    return 0

It takes one or more file paths and exec()s each file's contents. There are no other flags.

Caution

The CLI executes arbitrary Python via exec(). Never run diagrams on a .py file you do not trust — it has the same security profile as python <file>. This is a deliberate convenience for batch-rendering trusted diagram scripts in CI, not a sandbox. See Security.

5. The Auto-Generation Pipeline (Maintainer-Critical)

This is the part of the codebase that surprises newcomers: the provider node classes are generated, not written. Editing diagrams/aws/compute.py by hand is wrong — it will be overwritten. The pipeline lives in autogen.sh, config.py, scripts/, and templates/.

5.1 Inputs and outputs

5.2 autogen.sh — the orchestrator

The script (run from the repo root) does, for each provider in its providers array:

1. Preprocess resources depending on provider:
   • onprem / azure: convert SVG → PNG with Inkscape (python -m scripts.resource svg2png).
   • oci / ibm: convert SVG → PNG with ImageMagick (svg2png2).
   • All: clean resource file names (scripts.resource clean).
   • aws: round the PNG corners (scripts.resource round).
2. Generate modules & docs: python -m scripts.generate <provider> for every provider, plus custom.
3. Copy icons into the website static folder.
4. Lint the generated modules with black.

Important

autogen.sh requires external tools on the maintainer's machine: round, inkscape, convert (ImageMagick), and black. It exits early if any are missing. These are build-time only — end users need none of them.

5.3 config.py — the naming brain

config.py is the single source of truth for how an icon filename becomes a Python class name. Key tables:

It also defines the resource-processing commands (CMD_ROUND, CMD_SVG2PNG = Inkscape with 256×256 export, CMD_SVG2PNG_IM = ImageMagick) and the directory layout (DIR_DOC_ROOT="docs/nodes", DIR_APP_ROOT="diagrams", DIR_RESOURCE="resources", DIR_TEMPLATE="templates").

5.4 Generated module shape

Every generated category module follows the same pattern (here, AWS compute):

# This module is automatically generated by autogen.sh. DO NOT EDIT.
from . import _AWS

class _Compute(_AWS):
    _type = "compute"
    _icon_dir = "resources/aws/compute"

class EC2(_Compute):
    _icon = "ec2.png"
# ... one 3-line class per icon, plus alias assignments at the bottom

The provider's __init__.py defines the private _<Provider> base (setting _provider and the top-level _icon_dir); each category module defines a private _<Category> base that points _icon_dir at the category's resource folder; each node is a leaf class whose only body is _icon = "<file>.png". Aliases from config.py are appended as simple assignments (ECR = EC2ContainerRegistry).

6. Repository Layout

diagrams/
├── diagrams/                 # the importable package
│   ├── __init__.py           # ★ runtime core (Diagram/Cluster/Node/Edge)
│   ├── cli.py                # console entry point (exec diagram files)
│   ├── base/                 # internal base nodes
│   ├── custom/               # Custom (user-supplied icon) node
│   ├── c4/                   # C4-model helpers (hand-written)
│   ├── aws/  azure/  gcp/  k8s/  onprem/  ...   # AUTO-GENERATED provider modules
│   └── <17 more providers>
├── resources/                # source icon PNGs/SVGs (input to generation)
├── scripts/                  # generate.py, resource.py (the generators)
├── templates/                # Jinja2 templates (module.tmpl, apidoc.tmpl)
├── docs/                     # docusaurus site source (getting-started, guides, nodes)
├── website/                  # docusaurus site (v1) + copied icons
├── tests/                    # pytest suite (test_diagram, test_c4, test_cli)
├── autogen.sh                # ★ regenerate everything from resources/
├── config.py                 # ★ naming rules / aliases / prefixes
└── pyproject.toml            # hatchling build, deps, console script

The three ★ files plus the runtime __init__.py are where all the real logic lives; everything under the provider directories is derived.

7. Dependencies

System requirement: Graphviz must be installed separately (the graphviz PyPI package is only a binding, not the engine).

Note

Because the only hard runtime dependency that does work is the graphviz binding, Diagrams has a very small attack/maintenance surface. jinja2 matters only to maintainers running autogen.sh.

8. Security Considerations

Caution

Do not expose the diagrams CLI as a service that renders user-submitted files. exec() of untrusted input is remote code execution by design. If you must accept user diagram code, run it in a locked-down sandbox (container, seccomp, no network, read-only FS) — the library provides none of that itself.

9. Operations & CI

Diagrams has no runtime to "operate" — it's a library. The operational concerns are about rendering in automation:

• [ ] Install Graphviz on the runner (apt-get install graphviz / brew install graphviz / choco install graphviz).
• [ ] pip install diagrams into the same environment the scripts run in (avoid pipx isolation).
• [ ] Run each diagram with show=False (no viewer in headless CI).
• [ ] Render via python diagram.py or batch with diagrams *.py.
• [ ] Commit the .py sources; treat generated images as build artifacts (regenerate, don't hand-edit).

Regenerating provider modules (maintainers)

# requires: inkscape, imagemagick (convert), round, black on PATH
$ ./autogen.sh

starting
converting the svg to png using inkscape for provider 'onprem'
cleaning the resource names for provider 'aws'
rounding the resources for provider 'aws'
generating the modules & docs for provider 'aws'
...
linting the all the diagram modules

Tip

After autogen.sh, review the git diff under diagrams/<provider>/ to confirm only intended icons changed. A noisy diff usually means a resource rename or a new alias is needed in config.py.

10. Testing Strategy

The tests/ suite (pytest) covers the hand-written runtime, not the generated classes:

Dev tooling (from pyproject.toml): pytest, pylint, isort, black (line length 120), pre-commit, rope. The repo ships a .pre-commit-config.yaml running check-yaml, end-of-file-fixer, trailing-whitespace, seed-isort-config, isort, and autopep8 (aggressive, max-line-length 120).

Note

The generated provider classes are not individually unit-tested — they are mechanically identical, so testing the generator and the Node base is sufficient. If you change the template or config.py, regenerate and run the full suite plus a visual smoke test (render one diagram per provider).

11. Extending the Library

Add a node to an existing provider

1. Drop the icon PNG (or SVG, for providers that convert) into resources/<provider>/<category>/.
2. If the filename needs cleanup, add prefix/alias/casing rules to config.py (FILE_PREFIXES, UPPER_WORDS, TITLE_WORDS, ALIASES).
3. Run ./autogen.sh (or python -m scripts.generate <provider>).
4. Verify the new class appears in diagrams/<provider>/<category>.py and renders.

Warning

Never add the class by editing the generated file. Add the icon and (if needed) a config rule, then regenerate. Hand edits are lost on the next generation.

Add a whole provider

1. Add the provider name to PROVIDERS in config.py and the providers array in autogen.sh.
2. Populate resources/<provider>/<category>/... with icons.
3. Add any prefix/alias/casing entries for the provider in config.py.
4. Run ./autogen.sh. The provider's __init__.py, category modules, and docs are generated from the templates.

Add a hand-written abstraction (like C4)

C4 (diagrams/c4/__init__.py) is the model for hand-written, non-generated modules: it composes the core Node, Cluster, and Edge primitives into higher-level helpers (Person, System, Container, Database, SystemBoundary, Relationship) using HTML-style Graphviz labels. Follow this pattern for any new conceptual layer that isn't a flat icon catalog.

12. Known Issues, Limitations & Improvement Ideas

Drawn from the source (FIXME/TODO markers) and community-reported issues:

Build-quality wins

• The generator runs black on output but doesn't validate that every referenced icon exists — adding that check would have caught #1186.
• Provider modules are fully mechanical; a generation-time schema (provider → category → icon manifest) committed alongside the code would make diffs reviewable without re-running the toolchain.

13. Quick Reference