# Network Sketcher Administrator & Developer Guide

This guide is the operational and architectural reference for engineers
who run Network Sketcher in shared environments, integrate it with
surrounding systems, or extend it. It complements the User Guide: where
the User Guide answers *"how do I draw a diagram?"*, this guide answers
*"how does the engine work, what does every config knob do, what are
the failure modes, and where do I patch behavior?"*.

> [!NOTE]
> The repository ships **two editions** that share a common rendering
> engine. Operational concerns differ — the **Online** edition is what
> you deploy on a server; the **Offline** edition is what users run on
> their workstations. This guide is structured for the Online edition
> and calls out Offline differences explicitly where they matter.

---

## 1. Architecture Overview

Network Sketcher Online is a single-process Flask web service that
fronts a deterministic Python rendering engine. The engine is the same
code in both editions; the Online edition wraps it in headless stubs
(`nsm_adapter.py`) so it can run without a display server.

### 1.1 What it does

A user uploads a `[MASTER]*.xlsx` (or `.nsm`) file. The engine reads its
three well-known sheets (`Master_Data`, `Master_Data_L2`,
`Master_Data_L3`), decomposes them into in-memory section arrays, and
runs one of three dedicated diagram pipelines (L1, L2, L3) against them.
Output is written as PowerPoint (`.pptx`), SVG, or Excel
(`[DEVICE]*.xlsx`) and offered for download. A **CLI** layer
(`add`/`delete`/`rename`/`show`/`export`) lets users mutate the master
file by issuing line-oriented commands; mutations are versioned so the
original is preserved.

### 1.2 ASCII architecture

```text
                   ┌─────────────────────────────────────────────────┐
                   │              Browser (HTTPS, :5443)             │
                   │  Single-page UI · SSE · localStorage(job_id)    │
                   └────────────────────┬────────────────────────────┘
                                        │ TLS (self-signed by default)
                   ┌────────────────────▼────────────────────────────┐
                   │           ns_web_start.py (Flask app)           │
                   │ Routes: /upload /generate_step /run_commands    │
                   │   /svg_grid_stream /download(_visio|_drawio|all)│
                   │ Headers: HSTS · X-Frame-Options · CSP           │
                   │ Per-job state: uploads/<uuid>/                  │
                   │ Heartbeats: in-memory dict, 5-min idle expiry   │
                   └────┬───────────────┬───────────────┬────────────┘
                        │               │               │
        in-process run_cli              │   subprocess(_ns_worker.py)
                        │               │               │
                   ┌────▼───────────────▼───────────────▼────────────┐
                   │          ns_engine/  (headless engine)          │
                   │ nsm_adapter.py    Tk stubs + bootstrap          │
                   │ nsm_cli.py        verb dispatcher (5 verbs)     │
                   │ nsm_io.py         .xlsx ↔ .nsm (Parquet) bridge │
                   │ nsm_l1_*.py       L1 diagram + table + sync     │
                   │ nsm_l2_*.py       L2 diagram + table + sync     │
                   │ nsm_l3_*.py       L3 diagram + table + sync     │
                   │ nsm_ddx_*.py      pptx / svg renderers          │
                   │ nsm_extensions.py AI-context · flow · IP report │
                   │ nsm_sync_between_layers.py  L2↔L3 consistency   │
                   └────┬───────────────────────┬─────────────────────┘
                        │                       │
                ┌───────▼─────────┐    ┌────────▼───────────────────┐
                │ Master file     │    │ Output artifacts           │
                │ [MASTER]*.xlsx  │    │ [L1_DIAGRAM]*.pptx/.svg    │
                │  or *.nsm (zip  │    │ [L2_DIAGRAM]<area>_*       │
                │  of Parquet)    │    │ [L3_DIAGRAM]*              │
                │ ─ Master_Data   │    │ [DEVICE]*.xlsx             │
                │ ─ Master_Data_L2│    │ [AI_Context]*.txt          │
                │ ─ Master_Data_L3│    └────────────────────────────┘
                └─────────────────┘
```

### 1.3 Component inventory

| Component | Path | Role |
|---|---|---|
| Server entrypoint | `start_ns_online.py` / `stop_ns_online.py` | Daemonizes / kills `ns_web_start.py` cross-platform; manages PID file |
| Flask app | `network-sketcher_online/ns_web_start.py` | HTTPS server, request validation, security headers, daily access log, heartbeat eviction, parallel SVG generator |
| Engine bootstrap | `network-sketcher_online/ns_engine/nsm_adapter.py` | Headless Tk stubs (`_Noop`, `Entry`, `StringVar`...), openpyxl `.nsm`-load monkey-patch, in-process CLI runner (`run_cli`, `run_cli_isolated`) |
| Subprocess worker | `network-sketcher_online/ns_engine/_ns_worker.py` | Stand-alone Python that bypasses the in-process `_cli_lock` for true parallelism |
| CLI dispatcher | `network-sketcher_online/ns_engine/nsm_cli.py` (Online), `network-sketcher_offline/ns_cli.py` (Offline) | `add` / `delete` / `rename` / `show` / `export` verb routing, sub-command argument parsing |
| I/O adapter | `network-sketcher_online/ns_engine/nsm_io.py` | Reads/writes the 11 master sections from either `.xlsx` or `.nsm` (Parquet); type-preserves Python primitives via single-byte prefixes |
| L1 pipeline | `nsm_l1_diagram_create.py`, `nsm_l1_master_create.py`, `nsm_l1_table_from_master.py`, `nsm_l1_table_sync_master.py`, `nsm_l1_svg_create.py`, `nsm_l1_diagram_sync_master.py` | Layer-1 (physical wiring) diagram pipeline |
| L2 pipeline | `nsm_l2_diagram_create.py`, `nsm_l2_table_from_master.py`, `nsm_l2_table_sync_master.py`, `nsm_l2_svg_create.py` | Layer-2 (broadcast domain) per-area pipeline |
| L3 pipeline | `nsm_l3_diagram_create.py`, `nsm_l3_table_from_master.py`, `nsm_l3_table_sync_master.py`, `nsm_l3_svg_create.py` | Layer-3 (IP / VRF / VPN) pipeline |
| PPTX renderer | `nsm_ddx_figure.py` | `python-pptx` shape, line, segment, tag emitter |
| SVG renderers | `nsm_ddx_svg.py`, `nsm_ddx_svg_l2.py`, `nsm_ddx_svg_l3.py` | XML emitters for L1, L2, L3 SVG |
| Cross-layer sync | `nsm_sync_between_layers.py` | Keeps L1/L2/L3 tables consistent after a mutation |
| Extensions | `nsm_extensions.py` | AI-context export, flow report, IP report, auto-IP-addressing, summary diagram, CLI-on-GUI |
| Offline GUI | `network-sketcher_offline/network_sketcher.py` | Tkinter desktop application |
| External-system import | `network-sketcher_offline/ns_option_convert_to_master.py` | CML YAML, Visio SVG, draw.io SVG, NetBox CSV → master file |
| VPN diagrams | `network-sketcher_offline/ns_vpn_diagram_create.py` | L2 VPN overlay onto L1 (Offline only) |

### 1.4 Technology stack

| Layer | Tooling |
|---|---|
| Language | Python 3.9+ (3.11+ recommended) |
| Web framework | Flask (single-app, no blueprints) |
| TLS | Python `ssl` module with cipher pinning; self-signed cert generated via `cryptography` |
| Spreadsheet I/O | `openpyxl` (read/write) + `pandas` (intermediate frames) |
| Compact storage | `pyarrow` Parquet inside a ZIP (`.nsm`) |
| Presentation output | `python-pptx` |
| Graph layout | `networkx` for shortest-path / connected-components on flow-report and L2 broadcast-domain code paths |
| Numerics | `numpy` |
| Concurrency | `concurrent.futures.ThreadPoolExecutor`, `subprocess.run` (true parallel via separate interpreters), `threading.Lock` |

### 1.5 Data flow for the canonical "upload + render" workflow

```text
1. POST /upload (multipart) ──► validate_master_filename()
                              ► assign uuid4 job_id
                              ► save to uploads/<job_id>/[MASTER]*.xlsx
                              ► xlsx_to_nsm() → uploads/<job_id>/[MASTER]*.nsm
                              ► touch_heartbeat(job_id)
                              ◄ {job_id, filename}

2. POST /generate_step/<job_id>/init ──► run `show area`, `show device`,
                                          `show l1_link`, `show attribute`
                                          via run_ns_command()
                                       ► persist meta to .meta.json
                                       ◄ {areas, device_count, link_count,
                                          attribute_titles}

3. GET /svg_grid_stream/<job_id>     ──► fan out to ThreadPoolExecutor:
   (Server-Sent Events)                  - export l1_diagram --type all_areas_tag
                                          - export l1_diagram --type per_area_tag
                                          - export l3_diagram --type all_areas
                                          - export l3_diagram --type per_area
                                          - export l2_diagram --area <each>
                                       ► each task → run_ns_command_subprocess()
                                          (separate Python interpreter)
                                       ► moved files cached in _svg_mem_cache
                                       ◄ stream `data: {cell, files, svg_b64?}`
                                          for each completed cell
                                       ◄ background AI-context generation
                                          completes in parallel
                                       ◄ `data: {done: true, ai_filename}`

4. GET /download/<job_id>/<file>      ──► serve from uploads/<job_id>/

5. (heartbeat_timeout seconds idle)   ──► _cleanup_loop() rmtree(uploads/<job_id>/)
                                       ► drop _svg_mem_cache[job_id]
                                       ► drop _svg_grid_last_attr[job_id]
```

> [!NOTE]
> The engine is **deterministic**. A given master file at a given commit
> always produces the same byte-for-byte output. This is intentional:
> diff-friendly diagrams let teams review topology changes the same way
> they review source code, and it's why subprocess timeouts default to a
> generous four hours rather than failing fast — large topologies are
> bounded by RAM and CPU, not by stochastic behavior.

---

## 2. System Requirements

### 2.1 Minimum

| Resource | Floor |
|---|---|
| CPU | 2 cores (any x86_64 / ARM64 server-class CPU) |
| RAM | 4 GB |
| Disk | 1 GB free (source + dependencies + room for one session of upload artifacts) |
| Python | 3.9 (CPython) |
| OS | Linux (any modern distro), macOS 12+, Windows 10/11/Server 2019+ |
| Network | TCP `5443` inbound (configurable) |

### 2.2 Recommended for shared / production use

| Resource | Recommendation |
|---|---|
| CPU | 4–8 cores. `parallel_limit` defaults to `6` and benefits from real cores up to that count |
| RAM | 8–16 GB. L3 `all_areas` rendering for very large fabrics is memory-intensive |
| Disk | 10 GB free. Per-session `uploads/<job_id>/` directories can hold dozens of artifacts each |
| Python | 3.11+ for the speed-up over 3.9 in `pandas` / `pyarrow` paths |
| OS | Linux preferred (heartbeats and cleanup loop have been validated longest there) |

### 2.3 Python package floor

The exact pinned versions live in
`network-sketcher_offline/requirements_offline.txt` and
`network-sketcher_online/requirements_online.txt`. The functional floor:

| Package | Why |
|---|---|
| `flask` | Online server only |
| `openpyxl` | All editions (Excel I/O) |
| `pandas`, `pyarrow` | All editions (`.nsm` Parquet sections, I/O frames) |
| `python-pptx` | All editions (PowerPoint emitter) |
| `numpy` | All editions (layout math) |
| `networkx` | All editions (broadcast-domain detection, flow shortest-paths) |
| `cryptography` | Online edition (self-signed cert generation) |
| `tkinter`, `tkinterdnd2` | Offline GUI only |

> [!IMPORTANT]
> The Online edition does **not** require Tk. `nsm_adapter.bootstrap()`
> injects no-op stubs into `sys.modules['tkinter']`,
> `sys.modules['tkinter.ttk']`, `sys.modules['tkinter.filedialog']`, and
> `sys.modules['tkinter.messagebox']` before any engine module imports.
> This is how the Online edition runs cleanly in container images that
> have no display server.

---

## 3. Configuration Reference

This is the complete inventory of every configurable surface — JSON
file, environment variable, and CLI flag. If a setting is not in one of
these tables, it does not exist.

### 3.1 `ns_web_config.json` — the only config file

Located at `network-sketcher_online/ns_web_config.json`. Each top-level
key is `{"value": ..., "description": "..."}`; the loader strips the
description and keeps only the `value`. Keys whose names start with
underscore (`_description`) are ignored.

| Key | Type | Default | Description |
|---|---|---|---|
| `version` | string | `3.0.1h` | Display version; **not used by code**, kept for human inspection |
| `host` | string | `localhost` | Bind address. `0.0.0.0` for all NICs, or pin a specific IP |
| `fqdn` | string \| null | `null` | Certificate Common Name. If null, falls back to `host` |
| `port` | int | `5443` | HTTPS listen port |
| `max_file_size_mb` | int | `50` | Upload size cap (Flask `MAX_CONTENT_LENGTH`); over-cap returns HTTP 413 |
| `subprocess_timeout` | int \| null | `14400` | Per-subprocess wall clock cap in seconds; `null` = no limit |
| `heartbeat_timeout` | int | `300` | Seconds of inactivity before a session's `uploads/<job_id>/` is wiped |
| `cleanup_interval` | int | `60` | Seconds between idle-sweep runs in `_cleanup_loop` |
| `parallel_limit` | int | `6` | `max_workers` for the SSE-driven SVG generator |
| `ssl_cert_path` | string | `Certs/server.crt` | TLS cert path; relative to the config file or absolute |
| `ssl_key_path` | string | `Certs/server.key` | TLS private key path |
| `ssl_cert_validity_days` | int | `3650` | Validity window for auto-generated self-signed certs (10 years) |
| `ssl_key_size` | int | `4096` | RSA key bits for auto-generated certs |
| `ssl_min_version` | string | `TLSv1_2` | One of `TLSv1_2`, `TLSv1_3` |
| `ssl_max_version` | string | `TLSv1_3` | One of `TLSv1_2`, `TLSv1_3` |
| `ssl_ciphers` | string | `ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM` | OpenSSL cipher string |
| `log_level` | string | `INFO` | One of `DEBUG`, `INFO`, `WARNING`, `ERROR` |
| `access_log_retention_days` | int | `90` | Days of `access_*.log` to keep (purged on date roll-over and at startup) |
| `ai_context_button_label` | string | `Copy & Open LLM` | UI button label (templated into HTML at request time) |
| `ai_context_button_url` | string | `https://claude.ai` | URL the **Copy & Open LLM** button opens |
| `ai_context_show_commands` | array<string> | (15 `show ...` commands) | Show-command list the AI Context exporter runs in parallel — the order is preserved in the output file. Default contains `show area`, `show area_device`, `show area_location`, `show attribute`, `show device`, `show device_interface`, `show device_location`, `show l1_interface`, `show l1_link`, `show l2_broadcast_domain`, `show l2_interface`, `show l3_broadcast_domain`, `show l3_interface`, `show waypoint`, `show waypoint_interface` |

> [!IMPORTANT]
> `ssl_cert_validity_days` defaults to **3650** (10 years). For
> production, replace the auto-generated cert with one from your
> internal CA — most browsers cap manually-trusted self-signed certs at
> 1 year of acceptance. See Section 7.4.

### 3.2 Environment variables

| Variable | Read in | Default | Effect |
|---|---|---|---|
| `NS_WEB_SERVER` | `nsm_l2_diagram_create.py` line 47 (Online and Offline copies) | unset | When set to any non-empty string, engine code follows the headless / server code path. The Online server sets it explicitly in `ns_web_start.py`. |
| `NS_DEBUG_L3` | `nsm_l3_diagram_create.py` line 324 (Online and Offline copies) | `0` | Set to `1` to dump verbose L3-layout diagnostics to stdout. Useful for diagnosing L3 layout regressions; ship the captured stdout when filing issues. |

### 3.3 CLI flags (every `ns_cli.py` invocation)

#### 3.3.1 Top-level verbs

| Verb | Required | Notes |
|---|---|---|
| `show <object>` | `--master <path>` | Pure read. Output is a Python-array literal, suitable for `ast.literal_eval` |
| `add <object> ...` | `--master <path>` | Mutates the master. The Online server adds `--skip_sync` for all but the last syncable command in a batch |
| `delete <object> ...` | `--master <path>` | Mutates the master |
| `rename <object> ...` | `--master <path>` | Mutates the master; preserves all back-pointers |
| `export <object> ...` | `--master <path>` | Produces output artifacts (PPTX / SVG / Excel / TXT) |

#### 3.3.2 `export l1_diagram` flags

| Flag | Values | Default | Effect |
|---|---|---|---|
| `--master` | path | (required) | Path to `[MASTER]*.xlsx` or `[MASTER]*.nsm` |
| `--type` | `all_areas`, `all_areas_tag`, `per_area`, `per_area_tag` | `all_areas_tag` | Single combined view vs. one diagram per area; `_tag` variants include interface name labels |
| `--attribute` | string | (none) | Title from the master file's Attribute sheet to color-code devices by |
| `--format` | `pptx`, `svg` | `pptx` | Output format |

#### 3.3.3 `export l2_diagram` flags

| Flag | Values | Default | Effect |
|---|---|---|---|
| `--master` | path | (required) | Master file |
| `--area` | string | (required) | Area name (must exist in `Master_Data_L2`) |
| `--format` | `pptx`, `svg` | `pptx` | Output format |

#### 3.3.4 `export l3_diagram` flags

| Flag | Values | Default | Effect |
|---|---|---|---|
| `--master` | path | (required) | Master file |
| `--type` | `all_areas`, `per_area` | `all_areas` | Single combined view vs. one diagram per area |
| `--attribute` | string | (none) | Color-coding attribute title |
| `--format` | `pptx`, `svg` | `pptx` | Output format |

#### 3.3.5 Other `export` subcommands

| Subcommand | Flags | Effect |
|---|---|---|
| `device_file` | `--master` | Emits `[DEVICE]<basename>.xlsx` with normalized L1/L2/L3/Attribute sheets |
| `ai_context_file` | `--master`, `--accept-security-risk` | Emits `[AI_Context]<basename>.txt`. Requires either interactive `YES` confirmation or `--accept-security-risk` |
| `master_file_backup` | `--master` | Timestamped backup of the master file |
| `master_file_nodata` | `--master` | Empty-template master next to the input |

#### 3.3.6 Mutating-command behavior flags

| Flag | Read by | Effect |
|---|---|---|
| `--skip_sync` | All `add`/`delete` mutating subcommands | Skips the post-mutation L2/L3 sync. Used internally by the Online server to amortize sync cost across a batch — only the last syncable command in a run executes the sync |
| `--one_msg` | All `show` subcommands | Concatenates output as a single string (rather than a list of lines). Used by the AI-context exporter to capture each show command's output in one piece |

### 3.4 First-run-state files

| File | Created by | Purpose |
|---|---|---|
| `ns_online.pid` | `start_ns_online.py` | PID of the running server; consumed by `stop_ns_online.py` |
| `Certs/server.crt`, `Certs/server.key` | `ns_web_start.py` startup | Self-signed TLS cert auto-generated if missing |
| `logs/access_YYYYMMDD.log` | `_DailyFileHandler` | One file per local date; `/heartbeat/*` paths excluded |
| `logs/server.log` | `start_ns_online.py` | Detached stdout/stderr |
| `uploads/<uuid4>/` | `/upload` route | Per-job working directory; auto-deleted on heartbeat expiry |
| `uploads/<uuid4>/.active_master` | `set_active_master()` | Pointer to the currently-active master filename within the job dir (handles versioning) |
| `uploads/<uuid4>/.meta.json` | `/generate_step/init` | Cached `{areas, device_count, link_count, attribute_titles}` for `/restore` |

---

## 4. Deployment & Scaling

Network Sketcher Online is a single-process Flask app — there is no
master/worker pool, no separate cache process, no message broker. This
keeps deployment simple but bounds horizontal scaling to running more
than one instance behind a load-balancer with sticky sessions (necessary
because per-session state lives in `uploads/<job_id>/` on the local
filesystem).

### 4.1 Single-host deployment (typical)

```bash
# 1. Clone and install
git clone https://github.com/cisco-open/network-sketcher.git
cd network-sketcher
python -m pip install -r network-sketcher_online/requirements_online.txt

# 2. Optional: edit ns_web_config.json for your host/port
${EDITOR:-vi} network-sketcher_online/ns_web_config.json

# 3. Start
python start_ns_online.py
```

The server runs in the foreground of the Python process started by
`Popen`, but `start_ns_online.py` detaches it on Windows
(`DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP`) and creates a new POSIX
session on Linux/macOS (`start_new_session=True`).

### 4.2 Run as a systemd unit (Linux)

```ini
# /etc/systemd/system/network-sketcher.service
[Unit]
Description=Network Sketcher Online
After=network.target

[Service]
Type=forking
User=netsketcher
WorkingDirectory=/opt/network-sketcher
ExecStart=/opt/network-sketcher/.venv/bin/python /opt/network-sketcher/start_ns_online.py
ExecStop=/opt/network-sketcher/.venv/bin/python /opt/network-sketcher/stop_ns_online.py
PIDFile=/opt/network-sketcher/network-sketcher_online/ns_online.pid
Restart=on-failure

[Install]
WantedBy=multi-user.target
```

> [!TIP]
> Use a dedicated unprivileged user for the service. The Online server
> never needs root; reading the config and binding port `5443` work
> fine as a normal user (ports above 1024).

### 4.3 Containerized deployment

```dockerfile
FROM python:3.11-slim

WORKDIR /opt/network-sketcher
COPY . .
RUN pip install --no-cache-dir \
    -r network-sketcher_online/requirements_online.txt

# Persist Certs and uploads across container restarts
VOLUME ["/opt/network-sketcher/network-sketcher_online/Certs",
        "/opt/network-sketcher/network-sketcher_online/uploads",
        "/opt/network-sketcher/network-sketcher_online/logs"]

EXPOSE 5443
CMD ["python", "start_ns_online.py"]
```

### 4.4 Reverse proxy with TLS termination

If you front Network Sketcher with nginx/Caddy/Traefik that already
holds your real cert:

1. Set `host: 127.0.0.1` and let the reverse proxy bind public ports.
2. Provide the upstream's own cert via `ssl_cert_path` /
   `ssl_key_path`, **or** disable upstream TLS by replacing the
   `ssl.SSLContext` in `ns_web_start.py` with a plain HTTP listener
   (currently not exposed as a config flag — patch required).
3. Forward `X-Forwarded-For` so access logs report client IPs.

> [!NOTE]
> The default config bakes HSTS into every response with
> `max-age=31536000; includeSubDomains`. Remove that header if you're
> intentionally serving HTTP behind a TLS-terminating proxy without
> HTTPS upstream.

### 4.5 Horizontal scaling

Each session's working files live on **local** disk under
`uploads/<job_id>/`. To run more than one Online instance:

- Place a load balancer in front (HAProxy / nginx / your cloud LB).
- Configure **sticky sessions** keyed on a cookie or header so a given
  `job_id` always lands on the same backend.
- Keep `Certs/`, `uploads/`, and `logs/` on instance-local volumes —
  do **not** put `uploads/` on shared NFS unless you're willing to
  serialize all parallel writes.

### 4.6 Stopping cleanly

`stop_ns_online.py` finds every running `ns_web_start.py` process via
the OS process list (Windows: `wmic process where ...`, Linux/macOS:
`pgrep -f ns_web_start.py`) and falls back to the PID file as a
secondary source. It sends `SIGTERM` (POSIX) or `taskkill /F /T /PID`
(Windows), then waits up to 10 seconds for the process(es) to exit.

> [!WARNING]
> A SIGTERM during diagram generation aborts the in-flight subprocess.
> Sessions persist on disk and the user can re-issue the same generate
> request after the server is back up — but any partial output files in
> the job directory are not cleaned automatically (the next session's
> heartbeat-expiry sweep will eventually remove them).

---

## 5. Operations

### 5.1 Health check

There is no dedicated `/health` route. A useful liveness probe is the
root path:

```bash
curl -ksf https://127.0.0.1:5443/ -o /dev/null && echo OK
```

A readiness probe should additionally verify the cert paths exist and
are readable.

### 5.2 Daily access log

`logs/access_YYYYMMDD.log` (one file per local date) contains one line
per request:

```text
2026-04-28 14:07:31 198.51.100.42 POST /upload 200 upload="[MASTER]campus.xlsx"
2026-04-28 14:07:33 198.51.100.42 POST /generate_step/96d6b2a4-.../init 200
2026-04-28 14:07:35 198.51.100.42 GET /svg_grid_stream/96d6b2a4-... 200
2026-04-28 14:08:12 198.51.100.42 GET /download/96d6b2a4-.../[L1_DIAGRAM]AllAreasTag_campus.pptx 200 download="[L1_DIAGRAM]AllAreasTag_campus.pptx"
```

`/heartbeat/<job_id>` is suppressed from access logs (it would otherwise
fire every few seconds). Files older than `access_log_retention_days`
are deleted on startup and on each date roll-over.

> [!NOTE]
> Network Sketcher does **not** rotate or compress access logs itself.
> If you ship them off-host, run `logrotate` (or the equivalent) on
> `logs/access_*.log`; the in-app cleanup is best-effort, not a backup.

### 5.3 Heartbeat eviction

Every successful upload registers a heartbeat with `touch_heartbeat()`.
The browser front-end refreshes it every few seconds via
`POST /heartbeat/<job_id>`. The server's `_cleanup_loop` thread runs
every `cleanup_interval` seconds and rmtree-s any session whose
heartbeat is older than `heartbeat_timeout`. The same loop drops the
SVG memory cache and the per-session attribute cache.

### 5.4 Common operational checklists

#### After a clean install

- [ ] `python start_ns_online.py` starts without error
- [ ] `https://localhost:5443/` loads in a browser
- [ ] Upload a master file, wait for the SSE grid to populate
- [ ] Confirm `logs/access_YYYYMMDD.log` is being written
- [ ] Confirm `Certs/server.crt` was created (4096-bit RSA, 10-year
      validity)

#### After a config change

- [ ] `python stop_ns_online.py` reports `Stopped N process(es).`
- [ ] `python start_ns_online.py` reports a fresh PID
- [ ] First request after restart returns 200
- [ ] Loaded `parallel_limit`, `port`, etc. match what's now in
      `ns_web_config.json`

#### Before a TLS cert renewal

- [ ] Backup `Certs/server.crt` and `Certs/server.key`
- [ ] Drop in the new cert and key with the same paths (or update
      `ssl_cert_path` / `ssl_key_path`)
- [ ] Restart
- [ ] Confirm the new fingerprint with
      `openssl x509 -in Certs/server.crt -noout -fingerprint -sha256`

### 5.5 Backup strategy

| What | Why | How often |
|---|---|---|
| `network-sketcher_online/ns_web_config.json` | Config drift recovery | Whenever changed |
| `network-sketcher_online/Certs/` | Avoid forced re-trust on rebuild | Whenever cert rotates |
| Source repo at the running commit | Reproducibility | At deploy time |
| Master files (`[MASTER]*.xlsx`) and their backups | Topology data is the only irreplaceable artifact | Daily, off-host |

`uploads/<job_id>/` directories are **not** backup-worthy — they're
transient and auto-deleted.

### 5.6 Frequent cross-edition syncs

Both editions duplicate ~95% of the engine code (Offline `ns_*.py` and
Online `ns_engine/nsm_*.py`). When you backport a fix:

1. Patch the Online file under `ns_engine/`.
2. Diff against the Offline file under `network-sketcher_offline/` —
   the names match one-to-one (`ns_l2_diagram_create.py` ↔
   `nsm_l2_diagram_create.py`).
3. Apply the corresponding change to the Offline file.
4. Run `python ns_cli.py show area --master "[MASTER]demo.xlsx"` from
   each edition to confirm dispatcher integrity.

---

## 6. Performance

### 6.1 Where time goes

For a "first render after upload":

| Phase | Driver | Typical share |
|---|---|---|
| `xlsx_to_nsm` conversion | Master file size, sheet count | 5–15% |
| `show area`/`device`/`l1_link`/`attribute` (init) | Master file size | 5% |
| L1 `all_areas_tag` diagram | Device + link count, area count | 20–30% |
| L1 `per_area_tag` (one per area) | Per-area complexity | 10–20% |
| L3 `all_areas` diagram | Device + L3 interface count | 25–40% |
| L3 `per_area` | Per-area L3 density | 5–15% |
| L2 per area | L2 broadcast-domain density | 5–15% (× number of areas) |
| AI Context (parallel) | Master file size | runs concurrently with diagrams |

`/svg_grid_stream` parallelizes across `parallel_limit` workers; the
critical path is the slowest single subprocess. For most fabrics that's
the L3 `all_areas` render.

### 6.2 Tuning levers

| Lever | Effect | When to use |
|---|---|---|
| Raise `parallel_limit` | More concurrent diagrams | You have spare cores and RAM |
| Lower `parallel_limit` | Less RAM pressure | You're hitting OOMs / multiprocessing fallback errors |
| Raise `subprocess_timeout` (or `null`) | Bigger fabrics finish | You've sized RAM for the job and don't want timeouts |
| Lower `heartbeat_timeout` | Frees disk faster | Many short sessions, disk pressure |
| Raise `heartbeat_timeout` | Better UX for slow users | Few long sessions |
| Use `.nsm` directly | Skips xlsx→nsm conversion | Repeated renders against an unchanged master |
| Use `--type per_area` | Avoids the `all_areas` blow-up on huge fabrics | Hundreds of devices, dozens of areas |

### 6.3 SVG caching path

After a successful subprocess render, the moved file's bytes are
cached in `_svg_mem_cache[job_id][filename]` under
`_svg_mem_cache_lock`. Subsequent requests to `/svg_raw/<job_id>/...`
serve from this cache, bypassing both the disk and the Windows AV-lock
re-read. The cache is dropped on heartbeat expiry.

### 6.4 Known performance hazards

> [!WARNING]
> On Windows, antivirus engines (notably Defender) re-scan files
> immediately after they're moved between directories. The engine
> mitigates this with `_wait_readable()` (10-second wait with 150ms
> polling) before serving SVGs. If you see "File not readable after
> 10s" warnings in `server.log`, your AV is holding files longer than
> 10 seconds — exclude the `uploads/` directory from real-time scanning
> or raise the timeout in `_wait_readable`.

---

## 7. Security

Network Sketcher's outputs **are sensitive**. A network topology
artifact reveals device names, vendor inventory (via interface naming),
IP plans, VRF / VPN structure, and broadcast-domain layout. Treat
every generated file as `Internal` at minimum, and the AI Context
file as `Confidential`.

### 7.1 What's in the box at first start

| Defense | Implementation | Notes |
|---|---|---|
| TLS by default | Auto-generated 4096-bit RSA self-signed cert with 10-year validity | Replace with a real cert before exposing beyond localhost |
| TLS version pinning | `ssl_min_version: TLSv1_2`, `ssl_max_version: TLSv1_3` | TLS 1.0/1.1 disabled by default |
| Cipher pinning | `ssl_ciphers: ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM` | No RC4, no DES, no NULL |
| `Strict-Transport-Security` | `max-age=31536000; includeSubDomains` | Forces browsers to HTTPS for one year |
| `X-Frame-Options` | `DENY` (default), `SAMEORIGIN` for `/device_preview/<...>` | Prevents clickjacking; allows internal thumbnail iframes |
| `Content-Security-Policy` | `default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; ...; frame-ancestors 'none'` | XSS / data-exfil reduction |
| `X-Content-Type-Options` | `nosniff` | Stops MIME sniffing exploits |
| `Referrer-Policy` | `strict-origin-when-cross-origin` | Limits referrer leakage |
| Job ID validation | UUID4 regex `^[a-f0-9\-]{36}$` | Prevents path traversal via job id |
| Filename validation | Must start with `[MASTER]` and end in `.xlsx` or `.nsm` | Rejects upload-traversal attempts |
| Upload size cap | `max_file_size_mb` (default 50 MB) | Returns HTTP 413 over cap |
| Per-session isolation | UUID4 directories | One user can't read another's master file |

> [!CAUTION]
> The default deployment has **no authentication**. Anyone who can
> reach the listening port can upload, render, and download. **Never
> expose a default Network Sketcher Online directly to the internet.**
> Put it behind an auth proxy (oauth2-proxy, Pomerium, your SSO IdP),
> a VPN, or a Tailscale tailnet before exposing it beyond localhost.

### 7.2 Sensitive data classification

| Artifact | Sensitivity | Why |
|---|---|---|
| `[MASTER]*.xlsx` / `*.nsm` | **High** | Contains the entire network plan |
| `[L1_DIAGRAM]*` / `[L2_DIAGRAM]*` / `[L3_DIAGRAM]*` | **High** | Visualizes device names, interface IDs, VLAN/VRF mappings |
| `[DEVICE]*.xlsx` | **High** | Tabular form of all interfaces and IPs |
| `[AI_Context]*.txt` | **Highest** | Plain-text dump intended for an external LLM |
| `logs/access_*.log` | **Medium** | Source IPs and request paths but no payloads |
| `logs/server.log` | **Medium** | May contain stack traces with file paths |

> [!WARNING]
> The AI Context file is the highest-risk output. It is designed to
> be pasted into a hosted LLM, which means it will be transmitted to
> a third party and may be retained for training (per the LLM
> vendor's TOS). The CLI requires either an interactive `YES`
> confirmation or `--accept-security-risk`; the web UI displays the
> same risk text in the tooltip. **Audit any policy that allows
> unattended `--accept-security-risk` use.** See `nsm_extensions.py`
> for the export logic and the warning text.

### 7.3 Recommended access boundary

| Role | Should access | Should not access |
|---|---|---|
| Network engineer (read) | The web UI for their own organization's master files; downloaded diagrams under their own handling | Other tenants' master files; raw `uploads/` directory on the host |
| Network engineer (write) | The web UI to mutate their own master via the **CLI Commands** panel | The host filesystem directly; other engineers' active sessions |
| Sysadmin / SRE | The host filesystem (`/opt/network-sketcher`), `logs/`, `Certs/`, `ns_web_config.json` | Master file contents (can read them, but should treat as confidential by policy) |
| External / 3rd-party | Nothing (use a separate instance per tenant, or strong front-door auth) | All of the above |

### 7.4 Production TLS

The auto-generated cert is for development convenience. To replace it:

```bash
# Stop the server
python stop_ns_online.py

# Drop in CA-issued cert and key
cp /path/to/your/cert.pem  network-sketcher_online/Certs/server.crt
cp /path/to/your/key.pem   network-sketcher_online/Certs/server.key

# Or set custom paths in ns_web_config.json:
#   "ssl_cert_path": {"value": "/etc/letsencrypt/live/sketcher.example.com/fullchain.pem"}
#   "ssl_key_path":  {"value": "/etc/letsencrypt/live/sketcher.example.com/privkey.pem"}
#   "fqdn":          {"value": "sketcher.example.com"}

python start_ns_online.py
```

Verify:

```bash
openssl s_client -connect sketcher.example.com:5443 -servername sketcher.example.com < /dev/null 2>&1 | openssl x509 -noout -subject -dates
```

### 7.5 Secrets handling

Network Sketcher itself stores **no secrets** in code or config. The
only sensitive material on disk is:

- TLS private key (`Certs/server.key`) — chmod `0600`, owned by the
  service user.
- Master files in `uploads/<job_id>/` — auto-cleaned on heartbeat
  expiry. For long-lived hosting, consider mounting `uploads/` on an
  encrypted volume.

There are no API keys, no database credentials, no third-party tokens.
The `ai_context_button_url` defaults to a public LLM URL but is purely
a hyperlink — Network Sketcher itself never calls out to it.

### 7.6 Sanitization recommendations before sharing diagrams

If you need to share a diagram externally (vendor RFP, conference talk,
public docs):

1. Make a copy of the master and run a `rename` pass:
   - `rename device <real-name> <Device-N>` for every device
   - `rename area <real-area> <Area-N>` for every area
2. Run a `delete ip_address` pass on the L3 sheet (or set IPs to
   `RFC1918-only` representatives via `add ip_address`).
3. Re-render the diagrams from the sanitized master.
4. **Do not redact PNG/PPTX exports manually** — the underlying SVG
   may still contain the un-redacted text in `<text>` elements.

---

## 8. Developer Internals

### 8.1 Repository layout

```text
network-sketcher/
├── README.md                  # Cisco Open project README
├── CONTRIBUTING.md
├── CODE_OF_CONDUCT.md
├── SECURITY.md
├── LICENSE                    # Apache 2.0
├── start_ns_online.py         # Cross-platform daemonizer for Online
├── stop_ns_online.py          # Cross-platform stop
├── network-sketcher_offline/  # Tkinter GUI + standalone CLI
│   ├── network_sketcher.py    # GUI entrypoint
│   ├── ns_cli.py              # Standalone CLI dispatcher
│   ├── ns_def.py              # Shared helpers (file I/O, sheet→array, etc.)
│   ├── ns_dev.py              # GUI internals
│   ├── ns_extensions.py       # AI context, flow report, IP report, etc.
│   ├── ns_l1_*.py             # L1 pipeline (5 files)
│   ├── ns_l2_*.py             # L2 pipeline (3 files)
│   ├── ns_l3_*.py             # L3 pipeline (3 files)
│   ├── ns_ddx_figure.py       # PPTX renderer
│   ├── ns_egt_maker.py        # Excel template maker
│   ├── ns_attribute_table_*.py
│   ├── ns_option_convert_to_master.py  # External-system imports
│   ├── ns_sync_between_layers.py
│   ├── ns_vpn_diagram_create.py
│   ├── hook-pptx.py           # PyInstaller hook
│   ├── ns_logo.png
│   ├── requirements_offline.txt
│   └── Sample_data/
└── network-sketcher_online/   # Flask server + headless engine
    ├── ns_web_start.py        # Flask app: routes, security headers, parallelism
    ├── ns_web_config.json     # The only config file
    ├── help_msg.json          # Tooltip text for the web UI
    ├── requirements_online.txt
    ├── Certs/                 # Auto-populated on first start
    ├── logs/                  # Auto-populated on first request
    ├── static/                # JS deps (PptxViewJS, chart.umd, jszip, xlsx)
    └── ns_engine/             # Headless engine (mirror of offline ns_*)
        ├── __init__.py
        ├── _ns_worker.py      # Subprocess worker entry
        ├── nsm_adapter.py     # Tk stubs + bootstrap + run_cli
        ├── nsm_io.py          # xlsx ↔ nsm (Parquet) bridge
        ├── nsm_cli.py
        ├── nsm_def.py
        ├── nsm_l1_*.py / nsm_l2_*.py / nsm_l3_*.py
        ├── nsm_ddx_figure.py / nsm_ddx_svg*.py
        ├── nsm_extensions.py
        ├── nsm_svg_compute.py / nsm_svg_worker.py
        └── nsm_sync_between_layers.py
```

### 8.2 Key invariants

> [!NOTE]
> Master file integrity is preserved by design. The Online server
> never overwrites an uploaded master in place: any mutation creates a
> versioned copy (`[MASTER]<basename>_<n>.xlsx`) and updates
> `.active_master`. This is the single most important behavior to
> preserve when patching.

> [!NOTE]
> The engine is **deterministic with respect to a given master + flag
> combination**. If a refactor changes byte-for-byte output, that's a
> regression that needs review even if the diagrams *look* the same.
> Reviewers should diff the SVG output against a known-good master in
> `network-sketcher_offline/Sample_data/`.

### 8.3 Module reference

#### `ns_web_start.py` — Flask app

Major helpers:

| Helper | Purpose |
|---|---|
| `_load_config()` | Loads `ns_web_config.json`, strips description, returns flat dict |
| `_load_help_messages()` | Loads `help_msg.json` for the UI tooltip system |
| `validate_master_filename(filename)` | Enforces `[MASTER]*.xlsx`/`*.nsm` |
| `sanitize_job_id(job_id)` | Enforces UUID4 regex |
| `find_master_file(work_dir)` | Picks the first `[MASTER]*` in a job dir |
| `get_active_master(work_dir)` / `set_active_master(work_dir, name)` | `.active_master` pointer |
| `run_ns_command(args)` | In-process CLI run via `nsm_adapter.run_cli` |
| `run_ns_command_isolated(args, work_dir, master_filename)` | Same, but in a temp subdir copy |
| `run_ns_command_subprocess(args, work_dir, master_filename)` | Separate Python interpreter (true parallelism) |
| `_wait_readable(path, timeout=10.0)` | Windows AV-lock backoff |
| `generate_ai_context_parallel(work_dir, master_filename)` | Fans out the configured show commands across a thread pool |
| `_DailyFileHandler` | Per-day log file with auto-cleanup |
| `_cleanup_loop()` | Heartbeat-driven session sweep |
| `_convert_svg_for_visio(svg_bytes)` | SVG2 → SVG1.1 transformations for Visio |
| `_convert_svg_to_drawio(svg_bytes)` | SVG → `mxGraphModel` XML |

#### `nsm_adapter.py` — engine bootstrap

| Component | Purpose |
|---|---|
| `_Noop` | Catch-all stub class returning self for any attribute / call. Backs every Tk widget stub |
| `Toplevel`, `Frame`, `Label`, `Entry`, `Text`, `StringVar`, `IntVar`, `BooleanVar`, ... | One stub class per Tk widget the engine references |
| `_TtkStub` | Equivalent stubs for `tkinter.ttk` |
| `_FileDialogStub` | `askopenfilename`, `askopenfilenames`, `asksaveasfilename`, `askdirectory` (all return empty) |
| `_MessageBoxStub` | `showinfo`, `showwarning`, `showerror`, `askquestion`, `askokcancel`, `askyesno`, `askretrycancel` |
| `bootstrap()` | Idempotent: injects all stubs into `sys.modules`, sets up `sys.path`, monkey-patches `openpyxl.load_workbook` to handle `.nsm` files |
| `_patch_openpyxl_for_nsm()` | Transparent `.nsm` → temp-`.xlsx` translation, cached for the session |
| `run_cli(args, cwd)` | In-process CLI runner with global `_cli_lock` (single command at a time per process) |
| `run_cli_isolated(args, work_dir, master_filename)` | Temp subdir + master copy + run + move outputs |

#### `nsm_io.py` — master file I/O

| Helper | Purpose |
|---|---|
| `SECTION_SHEET_MAP` | The 11 master sections with their `(sheet_name, section_tag)` |
| `_encode_cell(value)` / `_decode_cell(v)` | Type-preserving encoder using single-byte prefix (`\x00I` int, `\x00F` float, `\x00B` bool) |
| `_read_section_from_xlsx(path, ws_name, section_tag)` | Section reader that locates `<<TAG>>` and walks until the next tag or 3000 empty rows |
| `xlsx_to_nsm(xlsx_path, nsm_path)` | ZIP of Parquet files |
| `nsm_to_xlsx(nsm_path, xlsx_path)` | Reverse |

#### `nsm_cli.py` — verb dispatcher

The dispatcher reads `argv_array` and routes by literal verb match
(`'show' in argv_array`, etc.), then by `next_arg` against the verb's
allow-list. The five subcommand allow-lists are documented in Section
3.3. Any unsupported verb or object prints the supported list and
exits.

#### Pipelines (`nsm_l1_*.py`, `nsm_l2_*.py`, `nsm_l3_*.py`)

Each layer has up to four files:

| Pattern | Role |
|---|---|
| `nsm_lN_master_create.py` (L1 only) | Builds the per-layer master from the high-level inputs |
| `nsm_lN_table_from_master.py` | Extracts the per-layer table view (shipped in `[DEVICE]*.xlsx`) |
| `nsm_lN_table_sync_master.py` | Keeps the per-layer table view consistent after master mutations |
| `nsm_lN_diagram_create.py` | Lays out the diagram and emits PPTX |
| `nsm_lN_svg_create.py` | Same layout, SVG emitter |
| `nsm_l1_diagram_sync_master.py` (L1 only) | Re-positions wires after device/area moves |

#### `nsm_extensions.py` — feature catalog

| Class | Purpose |
|---|---|
| `ai_context.export_ai_context_file(self, dummy)` | Writes `[AI_Context]<basename>.txt` from 15 `show` commands plus `nsm_extensions_cmd_list.txt` |
| `flow_report.create_device_flow_table(self, full_filepath_master, device_name)` | Per-device flow table with shortest-path routing (`networkx`) |
| `flow.add_routing_path_to_flow(...)` / `.get_shortest_path(...)` / `.append_flows_to_diagram(...)` / `.get_flow_item_list(...)` / `.export_flow_file(...)` | Flow-report internals |
| `ip_report.export_ip_report(self)` / `.get_folder_list(self)` | IP allocation report |
| `auto_ip_addressing.run_auto_ip(...)` / `.get_folder_list(...)` / `.get_auto_ip_param(...)` | Auto-assign IPs from a parameter set |
| `summary_diagram.export_summary_diagram(self)` | Single-slide topology summary |
| `cli_on_gui.execute_cli_commands(...)` / `.show_cli_input_window(...)` / `.show_command_output_window(...)` | Offline GUI CLI panel |

### 8.4 Code patterns and conventions

- **Tk in the engine?** All Tk imports go through stubs at module load
  via `nsm_adapter.bootstrap()`. The engine still calls `tk.Entry()`,
  `tk.StringVar()` etc. — those resolve to the `_Noop`-backed classes.
  Don't try to remove the Tk imports; they're required for shared
  code with the Offline edition.
- **CLI verb parsing:** simple membership checks (`'show' in
  argv_array`) followed by `get_next_arg(...)` lookups. There's no
  `argparse` use; if you add a flag, follow the existing
  `if a == '--master': replace_next = True` pattern.
- **Versioning:** never overwrite the input. Mutations write to
  `[MASTER]<basename>_<n>.xlsx` and update `.active_master`.
- **Headless detection:** check `os.environ.get('NS_WEB_SERVER')`
  rather than detecting Tk presence — the stubs make Tk presence
  always-truthy in Online mode.
- **Subprocess vs. in-process:** the Online server uses three
  execution modes: `run_ns_command` (in-process, holds `_cli_lock`),
  `run_ns_command_isolated` (in-process with master copy),
  `run_ns_command_subprocess` (separate interpreter for true
  parallelism). Pick the lowest-overhead one that satisfies your
  isolation needs.

### 8.5 Testing strategy

The repository ships **no formal test suite**. The maintainer-tested
verification path is:

1. `python ns_cli.py show area --master Sample_data/...` — sanity
   check that the dispatcher and master loader are wired up.
2. `python ns_cli.py export l1_diagram --master Sample_data/... --type
   all_areas_tag` — full L1 pipeline.
3. `python ns_cli.py export l2_diagram --master Sample_data/...
   --area <area>` — full L2 pipeline per area.
4. `python ns_cli.py export l3_diagram --master Sample_data/... --type
   all_areas` — full L3 pipeline.
5. End-to-end via the web UI for a final visual check.

When contributing, run all four CLI exports against
`Sample_data/No_data_Master_file_with_AI_Context/[MASTER]no_data.xlsx`
and visually diff the resulting `.pptx` against the previous version.

> [!TIP]
> Sample data is intentionally minimal — it's a *no-data* master with
> the AI-context placeholder. For more thorough testing, build a small
> synthetic master with two areas, four devices, and a handful of L1
> wires; commit it locally as a fixture.

### 8.6 Potential improvements

- **Test suite.** Even a smoke-test wrapping the four CLI exports
  would catch most regressions.
- **`/health` endpoint.** Currently health checks must hit `/`.
- **Auth integration hook.** The server has no auth surface; a
  pluggable middleware that delegates to JWT / OIDC / mTLS would
  make multi-tenant deploys safer.
- **Output retention policy.** `uploads/<job_id>/` currently lives
  for exactly `heartbeat_timeout` seconds; some users want
  "keep last 24 hours regardless of activity". Make the retention
  policy configurable separately from the heartbeat window.
- **Structured error response.** Routes return mixed
  `{success, message}` and `{error}` shapes. Standardizing would help
  programmatic clients.
- **Headless Tk removal.** Long-term, the engine would be cleaner if
  `Tk` references were stripped rather than stubbed. This is a large
  refactor (~25 modules) but enables packaging the engine as a library
  without the Adapter dance.

### 8.7 Contributing

The repository follows the Cisco Open contribution model:

- All code is Apache 2.0; every file carries the SPDX header
  `SPDX-License-Identifier: Apache-2.0` and `Copyright 2023 Cisco
  Systems, Inc. and its affiliates`.
- Open issues at
  [github.com/cisco-open/network-sketcher/issues](https://github.com/cisco-open/network-sketcher/issues).
- See `CONTRIBUTING.md` and `CODE_OF_CONDUCT.md` at the repo root.
- Security issues go to the address in `SECURITY.md`, **not** the
  public issue tracker.

---

## 9. Integration Points

Network Sketcher is intentionally narrow at its edges. It does **not**
talk to controllers, doesn't push configs, doesn't poll devices.
Integration happens through file imports and exports.

### 9.1 Inbound: external-system imports (Offline GUI)

The Offline GUI's **External Systems** tab imports four formats into a
master file via `ns_option_convert_to_master.py`:

| Format | Source | Class in code |
|---|---|---|
| YAML | Cisco CML topology export | `ns_option_convert_to_master_yaml` |
| SVG | Visio export | `ns_option_convert_to_master_svg` |
| SVG | draw.io export | `ns_option_convert_to_master_svg` (same class, dialect-detected) |
| CSV | NetBox device export | `ns_option_convert_to_master_csv` |

Two helper classes round out the import flow:
- `ns_overwrite_line_to_master_yaml` — overlays a YAML topology onto
  an existing master.
- `ns_l3_config_to_master_yaml` — adds L3 details from a YAML config
  bundle.

### 9.2 Outbound: derived export formats

| Format | Trigger | Implementation |
|---|---|---|
| `.pptx` | `export l1_diagram` / `l2_diagram` / `l3_diagram` (default `--format pptx`) | `nsm_ddx_figure.py` (python-pptx) |
| `.svg` | `--format svg` on diagram exports | `nsm_l1_svg_create.py` / `nsm_l2_svg_create.py` / `nsm_l3_svg_create.py` |
| Visio-friendly `.svg` | `GET /download_visio/<job_id>/<file>` | `_convert_svg_for_visio()` in `ns_web_start.py` |
| draw.io `.drawio` (XML) | `GET /download_drawio/<job_id>/<file>` | `_convert_svg_to_drawio()` in `ns_web_start.py` |
| `[DEVICE]<basename>.xlsx` | `export device_file` | Composed from L1/L2/L3/Attribute pipelines |
| `[AI_Context]<basename>.txt` | `export ai_context_file` | `nsm_extensions.ai_context.export_ai_context_file()` |
| `.nsm` (ZIP of Parquet) | Auto on Online upload | `nsm_io.xlsx_to_nsm()` |
| ZIP of all session artifacts | `GET /download_all/<job_id>` | Built in-memory by `download_all()` |

### 9.3 LLM integration

The AI Context flow is the only outbound call path:

1. The web UI (or `nsm_extensions.ai_context.export_ai_context_file`)
   builds a deterministic text bundle from the configured show
   commands plus `nsm_extensions_cmd_list.txt`.
2. The user clicks **Copy & Open LLM** (or hits the **Download**
   button). The button label and target URL are configured via
   `ai_context_button_label` and `ai_context_button_url`
   (default `https://claude.ai`).
3. The clipboard content is the AI Context plus the user's natural-
   language prompt.
4. The user pastes into the LLM, gets back CLI commands, pastes them
   into the **CLI Commands** box, and hits **Run**.

Network Sketcher itself never sends the data anywhere — the user
controls the transmission.

### 9.4 Programmatic engine usage

You can embed the engine in another Python program by bootstrapping
once and calling `run_cli()` repeatedly:

```python
from ns_engine.nsm_adapter import bootstrap, run_cli

bootstrap()

result = run_cli([
    'export', 'l1_diagram',
    '--master', '/path/to/[MASTER]campus.xlsx',
    '--type', 'all_areas_tag',
    '--format', 'svg',
])

print(result.returncode)   # 0 on success
print(result.stdout)        # captured stdout
print(result.stderr)        # captured stderr
```

`run_cli` holds a single global `_cli_lock`, so calls are serialized
within one process. For true parallelism, follow the pattern in
`run_ns_command_subprocess` and shell out to `_ns_worker.py` instead.

> [!TIP]
> If you're embedding the engine, prefer `run_cli_isolated()` over
> `run_cli()` for any mutating command — it copies the master into a
> temp directory first, so a crash in your code can't corrupt the
> caller's master file.

### 9.5 Front-end JS dependencies

The Online edition's UI uses these vendored JS libraries (under
`network-sketcher_online/static/`):

| File | Purpose |
|---|---|
| `PptxViewJS.min.js` | In-browser PowerPoint preview (zoom, pan) |
| `chart.umd.min.js` | Chart rendering for analytics panels |
| `jszip.min.js` | Client-side ZIP support for previews |
| `xlsx.full.min.js` | In-browser Excel preview |
| `logo.png` | Brand asset |

These are vendored to keep the deployment self-contained — no CDN
calls happen at runtime. If you upgrade a library, replace the file
in place and restart the server.

---

_Generated April 2026 from cisco-open/network-sketcher at ae4fafb. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._
