Flan Scan — Admin & Developer Guide

This guide is the operator's, architect's, and contributor's reference for Flan Scan, Cloudflare's lightweight network vulnerability scanner. It is the companion to the User Guide: where the User Guide says "run make start," this document explains what that actually does, why the container is shaped the way it is, what every module contributes, and how to extend or operate Flan Scan in production.

The codebase is deliberately small (32 files, ~26.5 KB of source) and cleanly layered: shell orchestration → Nmap with a CVE-matching script → Python XML parser → CVE description provider → pluggable report builder. This guide is organized around those layers and the operational concerns around them.

1. Executive Summary

Flan Scan is a stateless, containerized batch scanner. Each invocation reads a list of targets from /shared/ips.txt, runs one Nmap service-detection scan per target with the vulners NSE script, parses the resulting XML, enriches matched CVEs with descriptions, and renders a single report in one of four formats. There is no daemon, no queue, no API, and no persistent state beyond the files written to the mounted shared/ volume.

The design goal is operational simplicity: the unit of work is "scan these targets, produce one report," and every run is a fresh, disposable container.

2. Architecture

2.1 Component diagram

                         ┌─────────────────────────────────────────┐
                         │            Docker container              │
                         │           (python:3.9-alpine)            │
   /shared/ips.txt  ───► │                                          │
   (mounted volume)      │   run.sh  (ENTRYPOINT, orchestration)    │
                         │     │                                    │
                         │     ├─ for each IP:                      │
                         │     │    nmap -sV --script=vulners  ──────┼──► target hosts
                         │     │      → /shared/xml_files/<ts>/*.xml │
                         │     │                                     │
                         │     └─ python output_report.py           │
                         │            │                             │
                         │   ┌────────┴─────────────────────────┐   │
                         │   │ FlanXmlParser (contrib/parsers)  │   │
                         │   │   parses Nmap XML → ScanResult    │   │
                         │   └────────┬─────────────────────────┘   │
                         │            │                             │
                         │   ┌────────┴─────────────────────────┐   │
                         │   │ CveProjectProvider               │   │
                         │   │ (contrib/descriptions)  ──────────┼───┼──► CVEProject GitHub
                         │   │   CVE id → description + NIST url  │   │    + NIST NVD
                         │   └────────┬─────────────────────────┘   │
                         │            │                             │
                         │   ┌────────┴─────────────────────────┐   │
                         │   │ ReportBuilder (contrib/report_   │   │
                         │   │ builders): Latex│Markdown│Json│  │   │
                         │   │ JinjaHtml                        │   │
                         │   └────────┬─────────────────────────┘   │
                         │            ▼                             │
   /shared/reports/  ◄───┤   report_<ts>.<ext>                     │
   (mounted volume)      │            │                             │
                         │   upload() → aws_push.py / gcp_push.py ──┼──► S3 / GCS (optional)
                         └─────────────────────────────────────────┘

2.2 Component inventory

2.3 Data flow

1. run.sh reads /shared/ips.txt line by line.
2. For each line it runs nmap -sV -oX <xml> -oN - -v1 $@ --script=vulners/vulners.nse <ip>, writing XML to /shared/xml_files/<timestamp>/<ip>.xml.
3. After all targets are scanned, run.sh calls python /output_report.py <xml_dir> <report_file> /shared/ips.txt.
4. output_report.py selects a ReportBuilder from the format env var, parses every XML file with FlanXmlParser, and streams sections into the builder.
5. The builder's build() returns the rendered report, which is written to /shared/reports/report_<timestamp>.<ext>.
6. For LaTeX output, run.sh post-processes the file with sed to escape _, $, #, %.
7. If upload is set, run.sh calls aws_push.py or gcp_push.py for both XML and report.

3. System Requirements

Pinned language/runtime versions:

Warning

output_report.py and contrib/descriptions/cveproject.py import the requests library, but requests is not listed in requirements.txt. It is currently satisfied only as a transitive dependency of boto3/google-cloud-storage. See Potential Improvements.

4. Configuration Mastery

Flan Scan has no config file. All configuration is environment variables plus the ips.txt target list. The table below is the complete, source-based reference.

Target list — /shared/ips.txt: one IP or CIDR per line. Read by run.sh (to iterate scans) and by output_report.py (to list scanned IPs in the report). Must end with a newline or the final entry is skipped.

Important

The upload variable does more than toggle uploads. In run.sh, when upload is unset, root_dir=/shared/; when it is set, root_dir=/ and the script creates /xml_files and /reports at the container root. This is why the Kubernetes templates — which set upload — expect output at / rather than /shared.

5. Deployment

5.1 Docker

Build and run locally with the Makefile (see the Command Reference). The canonical run command is:

docker run --rm --cap-drop=all --cap-add=NET_RAW \
  --name flan_<timestamp> \
  -v "$(CURDIR)/shared:/shared:Z" \
  flan_scan

Key flags:

• --rm — container is disposable; removed on exit.
• --cap-drop=all --cap-add=NET_RAW — drop every Linux capability except raw-socket access.
• -v ...:/shared:Z — mount the working directory; :Z applies the SELinux relabel needed on SELinux hosts.

5.2 Kubernetes

The kubernetes_templates/ directory provides three manifests:

Both workloads mount two volumes:

• cloud-key — a Secret (cloudkey-key / cloud-key) mounted at /var/secrets/google for cloud credentials.
• shared-files — a ConfigMap mounting ips.txt at /shared.

Environment placeholders to fill in the manifests: upload (<GCP_OR_AWS>), bucket (<BUCKET_NAME>), format (<REPORT_FORMAT>), plus the container image reference.

Deploy steps:

• [ ] Add a local ips.txt path to kustomization.yaml.
• [ ] kubectl apply -k . to create the shared-files ConfigMap.
• [ ] kubectl get configmap to verify the ConfigMap exists.
• [ ] Set upload, bucket, format, image, and secrets in deployment.yaml (or cron_job.yaml).
• [ ] kubectl apply -f deployment.yaml (or cron_job.yaml).

Note

cron_job.yaml uses apiVersion: batch/v1beta1, which is removed in Kubernetes 1.25+. On modern clusters, change it to batch/v1. See Potential Improvements.

Caution

The CronJob and Deployment templates do not set --cap-drop=all --cap-add=NET_RAW — that hardening is applied only by the Makefile for local runs. When deploying to Kubernetes, add an explicit securityContext to drop all capabilities and add only NET_RAW.

6. Security

6.1 Runtime privilege

Local runs drop all Linux capabilities and add back only NET_RAW, which Nmap needs for raw-socket scan types (-sV service detection, SYN scans). This is a strong default: the container cannot, for example, change file ownership, load kernel modules, or bind privileged ports beyond what raw sockets allow.

6.2 Secrets handling

• Cloud credentials are supplied as environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) or a mounted key file (GOOGLE_APPLICATION_CREDENTIALS pointing at a JSON key, typically placed in shared/).
• In Kubernetes, credentials come from a mounted Secret at /var/secrets/google.

Warning

The GCS workflow instructs copying the service-account key into shared/ (cp key.json shared/). Because shared/ is a host-mounted volume, that private key now lives on the host filesystem and inside any report bucket if accidentally uploaded. Treat shared/ as sensitive: restrict its permissions and never commit key.json.

6.3 Input validation & network exposure

• No inbound surface. Flan Scan exposes no ports and serves no requests; it is a client that scans outward. There is no listener to attack.
• Target input (ips.txt) is trusted operator input passed to Nmap. Do not feed untrusted content into ips.txt, as extra Nmap flags can be injected via the container's argument passthrough ($@).
• Outbound calls go to CVEProject GitHub and NIST for descriptions; these are read-only GETs. A failure degrades gracefully (blank description) rather than aborting the report.
• Report content is derived from scan output and third-party CVE descriptions. LaTeX output escapes _, $, #, %; other formats rely on their libraries (Jinja2 autoescaping is enabled for HTML).

Important

Only scan networks you are authorized to scan. Flan Scan performs active service detection, which is intrusive and may be logged or flagged by target networks.

7. Operations

7.1 Start / stop

There is nothing to "stop" — each run is a one-shot container that exits when the report is written (--rm removes it). To stop a CronJob, delete or suspend it (kubectl patch cronjob flan-scan-job -p '{"spec":{"suspend":true}}').

7.2 Health & output verification

Success is observable by artifact:

• shared/xml_files/<timestamp>/ contains one XML per reachable target.
• shared/reports/report_<timestamp>.<ext> exists and is non-empty.

If the report is empty or missing services, inspect the raw XML to confirm Nmap detected open ports and versions.

7.3 Upgrades

Rebuild the image (make build, which is --no-cache) to pull the latest Nmap, nmap-scripts, and vulners script. There is no in-place upgrade path; the image is the unit of versioning.

7.4 Capacity

Scan wall-clock time scales with the number of targets and network latency, not local CPU. For large target sets, prefer splitting ips.txt across parallel CronJobs or hosts rather than one very long serial run (run.sh scans targets sequentially).

8. Performance Tuning

• Serial scanning: run.sh iterates targets one at a time. For large ranges, shard ips.txt and run multiple containers concurrently.
• Nmap flags: Pass timing templates (-T4) or reduced probes via the argument passthrough to speed up scans at some accuracy cost, e.g. docker run ... flan_scan -T4.
• Description caching: CveProjectProvider caches descriptions in-memory per run (self.cache), so repeated CVEs across services are fetched once. This cache does not persist between runs.
• Build caching: make build intentionally disables the Docker cache. If you need faster iterative builds during development, build with docker build directly (without --no-cache), accepting potentially stale vulnerability data.

9. Integrations

10. Developer Internals

10.1 Repository structure

flan/
├── run.sh                     # ENTRYPOINT orchestrator (scan loop + report + upload)
├── output_report.py           # Report pipeline entrypoint
├── aws_push.py / gcp_push.py  # Cloud upload helpers
├── Dockerfile / Makefile      # Packaging and run targets
├── requirements.txt           # Python deps
├── shared/
│   ├── ips.txt                # Target list
│   ├── reports/               # Output reports (gitignored)
│   └── xml_files/             # Raw Nmap XML (gitignored)
├── kubernetes_templates/      # Deployment, CronJob, Kustomization
└── contrib/
    ├── parsers/               # FlanXmlParser
    ├── internal_types/        # ScanResult, Vuln, SeverityLevels
    ├── descriptions/          # VulnDescriptionProvider, CveProjectProvider
    └── report_builders/       # ReportBuilder + 4 concrete builders + Jinja template

10.2 The report-builder pattern

The core design is a builder abstraction (ReportBuilder, contrib/report_builders/report_builder.py) with an ordered protocol that output_report.py:create_report() drives regardless of format:

init_report → add_vulnerable_section → initialize_section → add_vulnerable_services
            → add_non_vulnerable_section → initialize_section → add_non_vulnerable_services
            → add_ips_section → add_ip_address (per IP) → finalize → build

Concrete builders:

create_report_builder(report_type) in output_report.py maps format → builder via a dict; an unknown format raises NotImplementedError(report_type).

Tip

To add a new report format, subclass ReportBuilder (or JsonReportBuilder if you want the pre-structured buffer), implement the protocol methods, and add one entry to the builder_map dict in create_report_builder. No other file needs to change.

10.3 XML parsing (FlanXmlParser)

FlanXmlParser converts Nmap XML (via xmltodict) into ScanResult objects keyed by service/app name. Key behaviors verified in source:

• Handles both single-host and multi-host XML (list vs. dict) — a common Nmap XML shape difference.
• Skips ports whose state is closed; labels services with no name as unknown (guards the historical KeyError, issue #46).
• Parses vulnerabilities only from the vulners script output, and only the first time an app is seen (dedup by app name).
• get_app_name() builds a stable identifier from @product, @version, and CPE entries.

10.4 Types & severity

contrib/internal_types/flan_types.py defines:

• Vuln(name, vuln_type, severity) with to_dict() and CVSS→label mapping (convert_severity: <4 Low, <7 Medium, else High).
• ScanResult — per-service locations (ip → [ports]) and vulns list.
• SeverityLevels — string constants Low/Medium/High.

10.5 Description provider

CveProjectProvider (implements the abstract VulnDescriptionProvider) fetches a CVE's description from raw.githubusercontent.com/CVEProject/cvelist/... and pairs it with a NIST NVD URL. HTTP errors are caught and returned as a blank description with an error note, so report generation never aborts on a failed lookup.

10.6 Testing strategy

There is no test suite in the repository (no tests/ directory, no CI config in source). The abstraction seams — ReportBuilder and VulnDescriptionProvider are both abc.ABCMeta interfaces, and FlanXmlParser is pure (XML in, objects out) — make the code straightforward to unit-test. This is the single biggest contribution opportunity (see below).

11. Troubleshooting

Error: NotImplementedError: <format>

• Cause: create_report_builder was called with a format not in {tex, md, html, json} (output_report.py).
• Solution: Set format to a supported value.
• Prevention: Validate the format env var / manifest field against the four supported formats.

Error: mkdir: can't create directory '/shared/xml_files/...': Permission denied

• Cause: The mounted volume is not writable by the container (SELinux labeling or host permissions; issues #73, #10).
• Solution: Use the :Z mount flag (as the Makefile does) on SELinux hosts; ensure shared/, shared/reports/, and shared/xml_files/ are writable.
• Prevention: Deploy via the provided make targets or replicate their mount flags.

Error: Run-time Error: No Launch Config Set

• Cause: Container started outside the expected invocation pattern (issue #84).
• Solution: Use make start/make md|html|json or the documented docker run with /shared mounted and env vars set.
• Prevention: Standardize on the documented run commands / Kubernetes manifests.

Error: KeyError during report generation

• Cause: Historical parser crash on hosts that are down or return unexpected XML (issue #46).
• Solution: Rebuild from current source, which labels unknown services rather than crashing.
• Prevention: Keep the image rebuilt (make build).

Error: LaTeX report fails to compile

• Cause: Special characters in CVE summaries (issue #33). Flan escapes _, $, #, % but not all constructs.
• Solution: Use make md/make html/make json, or sanitize the offending summary.
• Prevention: Prefer non-LaTeX formats for automated pipelines.

Error: connection error / blank descriptions

• Cause: No outbound access to CVEProject/NIST (issue #52).
• Solution: Provide outbound HTTPS; blank descriptions are non-fatal.
• Prevention: Run in a network with egress, or accept missing descriptions.

Problem: Last target skipped

• Cause: ips.txt lacks a trailing newline (issue #21).
• Solution / Prevention: Ensure a trailing newline.

Error: free(): invalid pointer on make build

• Cause: Environment/architecture-specific build failure (issue #4).
• Solution: Rebuild; try a different host architecture / Docker version.
• Prevention: Build on a supported x86-64 host.

Problem: Uploads silently do nothing

• Cause: aws_push.py catches all upload exceptions and prints Error uploading to s3 without failing the run; a missing bucket or bad credential yields no uploaded file but a zero exit.
• Solution: Check container stdout for Error uploading to s3; verify bucket and credentials.
• Prevention: Validate credentials and bucket before scheduling.

12. Potential Improvements

Source-based tech-debt and enhancement opportunities:

• Missing requests dependency. requests is imported (output_report.py, cveproject.py) but absent from requirements.txt; it works only transitively. Add requests explicitly to avoid a break if boto3/google-cloud-storage drop it.
• Deprecated Kubernetes API. cron_job.yaml uses batch/v1beta1, removed in Kubernetes 1.25+. Migrate to batch/v1.
• No test suite. The clean interfaces (ReportBuilder, VulnDescriptionProvider, pure FlanXmlParser) are ideal for unit tests; none exist. Highest-value contribution.
• K8s manifests lack capability hardening. The Makefile drops capabilities, but deployment.yaml/cron_job.yaml do not. Add a securityContext mirroring --cap-drop=all --cap-add=NET_RAW.
• Silent upload failures. aws_push.py swallows exceptions; consider a non-zero exit or explicit logging on upload failure.
• Serial scanning. run.sh scans targets sequentially; large ranges would benefit from parallelism or sharding.
• Pinned, aging dependencies. boto3 1.12.15, google-cloud-storage 1.23.0, Jinja2 2.11.3 are old; review for security updates.

13. Quick Reference

13.1 Command Reference

13.2 Key Paths

13.3 Config Keys

13.4 Built-in Nmap command

nmap -sV -oX <xml-out> -oN - -v1 <passthrough-flags> --script=vulners/vulners.nse <ip>

13.5 Severity thresholds