Jesse A. Green / notebook

cheat.sh Admin Guide

Generated June 2026 from chubin/cheat.sh at 031a5d3

AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it.

A complete, source-based reference for operators, developers, and architects who want to run, extend, or understand cheat.sh — the community-driven cheat sheet aggregation service.

This guide covers the system architecture, every configuration parameter, the deployment model, security posture, operational procedures, and the internals needed to modify or self-host the service. For end-user query syntax, see the cheat.sh User Guide.


Table of Contents

  1. Executive Summary
  2. Architecture
  3. Request Lifecycle
  4. The Adapter System
  5. Routing Engine
  6. System Requirements
  7. Configuration Reference
  8. Deployment
  9. Operations
  10. Caching
  11. Security
  12. Repository Structure
  13. Module Reference
  14. Code Patterns & Conventions
  15. Testing
  16. Troubleshooting (Operator)
  17. Potential Improvements
  18. Quick Reference

1. Executive Summary

cheat.sh is a Python web service that presents a single curl/browser/editor interface to many community cheat sheet repositories and StackOverflow. Its core design idea is that the URL path is the query and the response is plain, pasteable text (or, for browsers, an HTML rendering of that same text).

Key technical characteristics:

  • Stateless HTTP service built on Flask, served in production by a gevent WSGI server for high concurrency.
  • Pluggable adapter architecture — each upstream source (tldr-pages, the cheat project, Learn X in Y Minutes, RosettaCode, StackOverflow, plus computed sources like RFC/OEIS/chmod) is a subclass of a common Adapter base.
  • Redis-backed caching keyed by topic_type:topic, with adapter-level control over what is cacheable.
  • Content-negotiated rendering — terminal user agents get ANSI text, browsers get HTML produced by piping the ANSI output through an ansi2html shell filter.
  • Self-hostable via Docker Compose (app container + Redis), with all cheat sheets fetched at image-build time.

Note

There is no database, no user accounts, and no authentication. State is limited to the Redis cache and optional per-client "last query" / session-id cookies. This keeps the service simple and horizontally scalable.


2. Architecture

High-level topology

                         ┌──────────────────────────────────────────┐
   curl / wget / xh ───► │  gevent WSGIServer  (bin/srv.py)          │
   browser          ───► │  ── wraps ──                              │
                         │  Flask app          (bin/app.py)          │
                         │    • User-Agent → ansi | html             │
                         │    • rate limiting (lib/limits.py)        │
                         │    • POST = cheat-sheet submission        │
                         └───────────────┬──────────────────────────┘
                                         │ cheat_wrapper(query, opts, fmt)
                                         ▼
                         ┌──────────────────────────────────────────┐
                         │  cheat_wrapper  (lib/cheat_wrapper.py)    │
                         │    parse query → route → postprocess      │
                         └───────────────┬──────────────────────────┘
                                         │ get_answers(topic)
                                         ▼
                         ┌──────────────────────────────────────────┐
                         │  Router  (lib/routing.py)                 │
                         │    routing table → topic_type(s)          │
                         │    cache lookup (lib/cache.py → Redis)    │
                         └───────────────┬──────────────────────────┘
                                         │ adapter.get_page_dict(topic)
                                         ▼
         ┌───────────────────────────────────────────────────────────────────┐
         │  Adapters  (lib/adapter/*)                                         │
         │  cheat.sheets · cheat · tldr · learnxiny · rosetta · question      │
         │  late.nz · fosdem · translation · rfc · oeis · chmod · internal    │
         └───────────────┬───────────────────────────────────────────────────┘
                         │ reads local git checkouts (upstream/) or runs commands
                         ▼
         ┌───────────────────────────────────────────────────────────────────┐
         │  Local cheat-sheet repositories (fetched by lib/fetch.py)         │
         │  + external upstream binary for StackOverflow questions           │
         └───────────────────────────────────────────────────────────────────┘
                                         │
                         ┌───────────────▼──────────────────────────┐
                         │  Frontends  (lib/frontend/*)              │
                         │   ansi.py  → Pygments-highlighted ANSI    │
                         │   html.py  → ansi2html.sh → HTML page      │
                         └──────────────────────────────────────────┘

Component inventory

Component File(s) Responsibility
Production server bin/srv.py gevent WSGI server, monkey-patches stdlib, binds host:port
Web application bin/app.py Flask routes, User-Agent detection, rate limiting, POST handling, logging
Wrapper lib/cheat_wrapper.py Parse/normalize the query, dispatch to router and search, postprocess
Router lib/routing.py Map a topic to one or more adapters via the routing table; manage cache
Adapters lib/adapter/*.py Retrieve answers from each source
Frontends lib/frontend/*.py Render answer data as ANSI or HTML
Formatting lib/fmt/*.py Comment beautification, internal-page colorization, markdown
Coloring lib lib/panela/*.py Internal ANSI/color manipulation library (panela)
Config lib/config.py Layered configuration (defaults → files → env)
Cache lib/cache.py Redis (or no-op) cache
Rate limiting lib/limits.py Per-IP query caps
Fetch/update lib/fetch.py Clone and update upstream cheat-sheet repos; invalidate cache
Standalone lib/standalone.py Offline command-line entry point (no server, no cache)
Client share/cht.sh.txt The cht.sh shell client served at /:cht.sh

Note

The service code lives in lib/ and bin/. The share/ directory holds the client script, shell completions, editor helpers, static assets, and the ansi2html.sh converter used for HTML rendering.


3. Request Lifecycle

A GET /<topic> request flows through the following steps (bin/app.py:answer):

  1. User-Agent inspection. _is_html_needed() checks the User-Agent against PLAIN_TEXT_AGENTS (curl, wget, httpie, lwp-request, python-requests, openbsd ftp, powershell, fetch, aiohttp, xh). A match means ANSI output; anything else (a browser) means HTML.
  2. Option parsing. options.parse_args(request.args) turns query-string flags into an options dict (see §7 → request options).
  3. Special early exits. Favicon/apple-touch icons return empty; :last returns the caller's previous query (requires a session-id cookie); POST requests are routed to process_post_request() (cheat-sheet submission).
  4. Language negotiation. get_answer_language() derives a human language from the *.cheat.sh subdomain, a ?lang= argument, or the Accept-Language header.
  5. Rate limiting. If the topic contains + (a multi-word query), Limits.check_ip() is consulted; over-limit requests get 429.
  6. Dispatch. cheat_wrapper(topic, request_options, output_format) runs the query and returns (result, found).
  7. Rendering & logging. The result is returned as text/plain (ANSI clients) or HTML (browsers). Every query is appended to the query log.

Inside cheat_wrapper() (lib/cheat_wrapper.py):

  1. Sanitize the query (strip < > ").
  2. Add section name — turn the first space/+ into a / so python lambda becomes python/lambda (with care for g++ and friends).
  3. Rewrite aliases & editor section names (emacs:go-mode → go, c++ → cpp).
  4. Parse for a ~keyword search and search options.
  5. Resolve — find_answers_by_keyword() for searches, otherwise get_answers() (the router).
  6. Postprocess each answer (add language-appropriate comments, filter by keyword).
  7. Visualize via the chosen frontend (ansi, html, or json).

Important

The json output format exists in cheat_wrapper() but bin/app.py only ever requests html or ansi. There is no JSON HTTP endpoint. Treat answers as text or HTML only.


4. The Adapter System

Every source of answers is an adapter — a subclass of Adapter (lib/adapter/adapter.py). This is the central extensibility mechanism.

Base class contract

Method Purpose
_get_list(prefix=None) Return the list of topics this adapter serves
_get_page(topic, request_options) Return the answer text (or dict) for a topic
is_found(topic) Whether this adapter has the topic (root only, by default)
is_cache_needed() Whether answers should be cached (_cache_needed)
get_page_dict(topic, ...) Wraps _get_page into the standard answer dict
local_repository_location() Where the adapter's git checkout lives
fetch_command() / update_command() Shell commands to clone/update the source

Adapter discovery is automatic: all_adapters() walks Adapter.__subclasses__() recursively, so adding a new adapter is as simple as defining a subclass in lib/adapter/ and adding its name to adapters.active in config.

Adapter taxonomy

Adapter (abstract)
├── RepositoryAdapter            file-backed cheat sheets
│   └── GitRepositoryAdapter     adds git clone/pull/state for cache invalidation
│       ├── CheatSheets          chubin/cheat.sheets        (native sheets)
│       │   └── CheatSheetsDir   directory listings (LANG/)
│       ├── Cheat                cheat/cheatsheets
│       ├── Tldr                 tldr-pages/tldr            (.md, multi-platform)
│       ├── LearnXinY            adambard/learnxinyminutes  (:learn pages)
│       ├── Rosetta              acmeism/RosettaCodeData
│       └── Latenz               chubin/late.nz            (curlable latencies)
├── CommandAdapter               answer = stdout of a shell command
│   ├── Fosdem                   current FOSDEM slide (demo)
│   ├── Translation              external get_translation binary
│   ├── AdapterRfc               share/adapters/rfc.sh
│   ├── AdapterOeis              share/adapters/oeis.sh
│   └── AdapterChmod             share/adapters/chmod.sh
├── UpstreamAdapter              proxy to another cheat.sh server
│   └── Question                 StackOverflow via external upstream binary
├── InternalPages                :help, :intro, :list, :styles, etc.
│   └── UnknownPages             fuzzy "did you mean…" suggestions
└── Search                       marker adapter for the search route

Notable adapters

  • CheatSheets — the native sheets from chubin/cheat.sheets, hierarchically organized; directory names are stored with a leading _ that is stripped for queries.
  • Tldr — reads tldr-pages .md files, searching common, linux, osx, sunos, windows, android in order, and applies a trivial Markdown transform.
  • LearnXinY — parses each language's *.html.markdown file from Learn X in Y Minutes into blocks; powers the :learn pages. Each language has its own subclass with block-separator regexes.
  • Question — answers free-form programming questions. It shells out to an external binary at path.internal.bin.upstream; if that binary is absent, it falls back to UpstreamAdapter, proxying to upstream.url (default https://cheat.sh). It also runs language detection (polyglot) to support non-English questions.
  • UpstreamAdapter — proxies to another cheat.sh instance; on connection failure it returns the "Are you offline?" ASCII banner and marks the answer non-cacheable.

Tip

When self-hosting without the proprietary StackOverflow upstream binary, the question adapter will transparently proxy to the public https://cheat.sh. Set upstream.url to your own mirror, or accept the public default.


5. Routing Engine

Router (lib/routing.py) decides which adapter(s) answer a topic, using a routing table assembled from three configurable lists plus a default:

routing_table = routing.pre + routing.main + routing.post

Each entry is (regexp, adapter_name). get_topic_type() walks the table; every regex that matches and whose adapter reports is_found(topic) contributes a route. If more than one matches, the trailing (default) route is dropped. If none match, routing.default (question) is used.

Default routing table (from lib/config.py)

routing.pre (highest priority):

Regex Adapter Matches
^$ search Empty query
^[^/]*/rosetta(/|$) rosetta …/rosetta paths
^rfc/ rfc rfc/NNN
^oeis/ oeis oeis/ANNN
^chmod/ chmod chmod/NNN
^: internal Special pages (:help, …)
/:list$ internal Any …/:list
/$ cheat.sheets dir Trailing slash → directory listing

routing.main (the cheat-sheet sources, tried in order): cheat.sheets, cheat, tldr, late.nz, fosdem, learnxiny.

routing.post:

Regex Adapter Matches
^[^/ +]*$ unknown A single bare word with no match → "did you mean"
^[a-z][a-z]-[a-z][a-z]$ translation Locale pairs like en-de

routing.default: question (StackOverflow).

Special routing behaviors

  • :random — handle_if_random_request() rewrites /LANG/:random to a random real topic under that prefix before routing.
  • Explicit type — a topic of the form type:topic (e.g. tldr:tar) forces a single adapter.
  • question caching — question answers use a legacy q:<topic> cache key and are reused even across cache-format changes, because they are expensive to generate.

Note

The set of live adapters is adapters.active ∪ adapters.mandatory. To disable a source, remove it from adapters.active; to add one, add its name there (and ship the adapter class).


6. System Requirements

Runtime

Requirement Detail
Python 3.x (the code carries some Python-2 compatibility shims but targets 3)
Redis For caching (cache.type: redis); optional if cache.type: none
git To fetch/update upstream cheat-sheet repositories
bash + gawk + sed Required by ansi2html.sh for HTML rendering
rlwrap Only on clients, for cht.sh --shell

Python dependencies (requirements.txt)

Package Role
gevent Async WSGI server (production)
flask Web framework / routing
requests Upstream proxying (Question/Upstream adapters)
pygments Syntax highlighting + the list of color styles
redis Cache backend
fuzzywuzzy / (optional rapidfuzz) "Did you mean" fuzzy topic matching
langdetect, polyglot, PyICU, pycld2 Human-language detection for questions
colored, colorama Terminal color handling
pyyaml Config + RosettaCode metadata parsing
dateutils, cffi, python-Levenshtein Supporting libraries
wheel, black, pytest Build / format / test tooling

Warning

polyglot + PyICU + pycld2 have native build dependencies (ICU, a C++ toolchain). The Dockerfile installs these via Alpine packages and a temporary build toolchain; expect non-trivial build steps if installing on bare metal.


7. Configuration Reference

Configuration is centralized in lib/config.py as a single CONFIG dictionary with dotted keys. Paths are made absolute relative to path.workdir.

Resolution order (highest wins)

  1. Environment variables (CHEATSH_*)
  2. Config file in the work dir — $WORKDIR/etc/config.yaml (or $CHEATSH_CONFIG)
  3. Config file in the project dir — <project>/etc/config.yaml
  4. Built-in defaults in _CONFIG

The work dir defaults to $HOME/.cheat.sh and can be overridden with CHEATSH_PATH_WORKDIR.

Environment-variable mapping

A config key maps to an env var by: prefix with CHEATSH_, replace . with _, uppercase. Only scalar (int/string) values can be set this way.

Config key Environment variable
server.port CHEATSH_SERVER_PORT
server.bind CHEATSH_SERVER_BIND
cache.type CHEATSH_CACHE_TYPE
cache.redis.host CHEATSH_CACHE_REDIS_HOST
cache.redis.port CHEATSH_CACHE_REDIS_PORT
cache.redis.db CHEATSH_CACHE_REDIS_DB
cache.redis.prefix CHEATSH_CACHE_REDIS_PREFIX
log.level CHEATSH_LOG_LEVEL
upstream.url CHEATSH_UPSTREAM_URL
upstream.timeout CHEATSH_UPSTREAM_TIMEOUT
search.limit CHEATSH_SEARCH_LIMIT

Caution

Two easy mistakes:

  1. CHEATSH_PORT is special-cased in bin/srv.py and read directly — it is not the same path as the config mapping. It takes precedence over server.port when set. CHEATSH_SERVER_PORT works too, via config.
  2. The env var must match the full dotted key. CHEATSH_REDIS_PREFIX (seen in tests/run-tests.sh) does not map to anything — the correct variable is CHEATSH_CACHE_REDIS_PREFIX.

Full default configuration

Key Default Meaning
adapters.active tldr, cheat, fosdem, translation, rosetta, late.nz, question, cheat.sheets, cheat.sheets dir, learnxiny, rfc, oeis, chmod Enabled sources
adapters.mandatory ["search"] Always-on adapters
cache.type redis redis or none
cache.redis.host localhost Redis host
cache.redis.port 6379 Redis port
cache.redis.db 0 Redis database number
cache.redis.prefix "" Optional key prefix
frontend.styles (all Pygments styles) Allowed ?style= values
log.level 4 Verbosity (≥5 prints extra adapter debug)
path.repositories upstream Where upstream repos are cloned
path.spool spool Where POSTed cheat sheets are stored
path.workdir $HOME/.cheat.sh Base work dir for all relative paths
path.log.main log/main.log Application log
path.log.queries log/queries.log Per-query log
path.log.fetch log/fetch.log Fetch/update log
path.internal.* (derived from project dir) Static assets, pages, templates, ansi2html.sh, upstream binary
routing.pre / routing.main / routing.post / routing.default (see §5) Routing table
upstream.url https://cheat.sh Fallback/proxy server
upstream.timeout 5 Upstream request timeout (seconds)
search.limit 20 Max answers returned for a search
server.bind 0.0.0.0 Bind address
server.port 8002 Listen port

etc/config.yaml (shipped example)

server:
    address: "0.0.0.0"
cache:
    type:   redis

Warning

The shipped etc/config.yaml uses the key server.address, but the real configuration key is server.bind. server.address does not map to any known key and is silently ignored (the default 0.0.0.0 is used regardless). When editing config, use the keys from the table above, not the example file's naming.

Request (per-query) options

Parsed by lib/options.py from the query string:

Letter Effect (internal flag)
c add_comments=False, unindent_code=False
C add_comments=False, unindent_code=True
Q remove_text=True (code only)
q quiet=True (suppress HTML chrome)
T no-terminal=True (strip ANSI colors)

Defaults: add_comments=True. Key-value args like style=native and lang=de pass through directly.


8. Deployment

The simplest self-hosted setup uses the bundled docker-compose.yml:

docker-compose up

This builds the app image, fetches every cheat sheet into the image at build time, and starts two services:

services:
  app:
    build: .
    image: cheat.sh
    container_name: chtsh
    depends_on:
      - redis
    environment:
      - CHEATSH_CACHE_REDIS_HOST=redis
    ports:
      - "8002:8002"
    volumes:
      - .:/app:Z
  redis:
    image: redis:4-alpine
    volumes:
      - redis_data:/data

The service is then available at http://localhost:8002.

Important

The README explicitly notes the Docker setup is an early implementation, recommended for internal/dev/personal use rather than large public production.

The image build (Dockerfile)

Key build stages (Alpine 3.14 base):

  1. Install runtime packages: git, py3-six, py3-pygments, py3-yaml, py3-gevent, libstdc++, py3-colorama, py3-requests, py3-icu, py3-redis, sed.
  2. Copy the source into /app.
  3. Install remaining Python deps via a temporary build toolchain (py3-pip, g++, python3-dev, libffi-dev), then remove it.
  4. Fetch all cheat sheets at build time: python3 lib/fetch.py fetch-all (into /root/.cheat.sh/).
  5. Install server packages: py3-jinja2, py3-flask, bash, gawk.
  6. Entry point: python3 -u bin/srv.py.

Debug deployment

docker-compose.debug.yml overrides the app to run the Flask dev server (with autoreload and tracebacks) instead of gevent:

docker-compose -f docker-compose.yml -f docker-compose.debug.yml up

It sets FLASK_ENV=development, FLASK_APP=bin/app.py, FLASK_RUN_HOST=0.0.0.0, FLASK_RUN_PORT=8002.

Caution

Never expose the Flask dev server (docker-compose.debug.yml) to the public internet. It is single-threaded, not hardened, and prints tracebacks. Use bin/srv.py (gevent) for anything reachable.

Running without Docker

# 1. Install dependencies
pip3 install -r requirements.txt

# 2. Fetch all upstream cheat sheets
python3 lib/fetch.py fetch-all

# 3. Start the production server (gevent)
python3 bin/srv.py                 # binds server.bind:server.port (0.0.0.0:8002)
CHEATSH_PORT=8080 python3 bin/srv.py   # override the port
python3 bin/srv.py --debug         # enable Flask debug (limited under gevent)

A reverse proxy (nginx, Traefik, …) typically terminates TLS in front of bin/srv.py. The app reads X-Forwarded-For for the client IP, so configure your proxy to set it.


9. Operations

Fetching and updating cheat sheets

All cheat-sheet content lives in cloned upstream repos under path.repositories (upstream/ in the work dir). Manage them with lib/fetch.py:

python3 lib/fetch.py fetch-all     # initial clone of every configured repo
python3 lib/fetch.py update-all    # git pull each repo + invalidate changed cache entries
python3 lib/fetch.py update NAME   # update one adapter's repo

update-all is the routine maintenance command: for each git-backed adapter it pulls, computes the changed files since the last recorded state (.cached_revision), and invalidates exactly those cache entries so updated sheets are re-rendered while everything else stays warm.

Tip

Schedule python3 lib/fetch.py update-all (e.g. via cron) to keep content fresh. Because it invalidates only changed entries, it is cheap to run often.

Logs

Log Path (relative to work dir) Contents
Main log/main.log Application/Flask logging (DEBUG level)
Queries log/queries.log One line per query: IP found topic user_agent
Fetch log/fetch.log Repository fetch/update activity

Flask exceptions are additionally sent to stderr (with werkzeug duplicate-line filtering), so container logs surface errors.

Cache maintenance

# delete specific Redis keys directly
python3 bin/clean_cache.py "tldr:tar" "cheat.sheets:python/lambda"

To flush everything, use redis-cli FLUSHDB against the configured database (be aware this also drops session/last-query keys).

Submitting cheat sheets (spool)

POSTed cheat sheets (:post) are written to path.spool as topic.name.<nonce> files for later review/merge. Monitor and drain this directory as part of operations.


10. Caching

Caching is implemented in lib/cache.py and controlled per-adapter via is_cache_needed().

Key scheme

Key pattern Used for
<topic_type>:<topic> Standard cached answers (e.g. tldr:tar)
q:<topic> StackOverflow question answers (legacy, expensive)
l:<client_id> A client's last query (for :last)

An optional global prefix (cache.redis.prefix) is prepended as <prefix>:.

Behavior

  • Values that are dict/list are JSON-serialized on put and parsed on get.
  • Adapters that set _cache_needed = True (tldr, cheat, learnxiny, rosetta, translation, rfc, oeis, chmod, question) are cached; others (cheat.sheets, internal, search, upstream failures) are not.
  • An adapter's _get_page() may override caching per-answer by returning a dict with a "cache" key (the UpstreamAdapter marks offline answers "cache": False).

Note

Switching cache.type to none (as the standalone CLI and the test harness do) disables Redis entirely — every request is computed fresh. Useful for testing or a no-Redis deployment, at the cost of latency for expensive sources.


11. Security

cheat.sh has a deliberately small attack surface (no auth, no DB, no user data), but operators should understand the following.

Input handling

  • Query sanitization — cheat_wrapper._sanitize_query() strips < > " from the query before processing, reducing reflected-content risk in the HTML page.
  • Command adapters run shell programs. CommandAdapter, AdapterRfc, AdapterOeis, and AdapterChmod execute helper scripts via subprocess with argument lists (not shell strings), and the chmod/oeis adapters scrub their topic input with regexes ([^a-zA-Z0-9-] etc.) before passing it. This avoids shell injection through the topic.
  • The fosdem and translation adapters invoke fixed binaries. fosdem uses a sudo NOPASSWD entry for a single demo command, and translation references a hard-coded path (/home/igor/cheat.sh/bin/get_translation). These are environment-specific and inert on a default self-host.

Rate limiting

lib/limits.py caps queries per source IP for multi-word queries (those containing +):

Interval Limit
Minute 30
Hour 600
Day 1000

Over-limit requests receive 429 Not so fast!. There is a hard-coded whitelist (_WHITELIST = ["5.9.243.177"]). The client IP is taken from X-Forwarded-For (first value) when present, else remote_addr.

Warning

Because the IP comes from X-Forwarded-For, only trust it behind a proxy you control. If bin/srv.py is directly internet-facing, a client can spoof the header and bypass rate limits. Always front the service with a reverse proxy that sets X-Forwarded-For from the real connection.

The :shell-x proxy

bin/app.py contains a _proxy() path for /:shell-x/ that forwards to a local service on port 3000. This is a special/experimental feature and a known source of 500 errors (see issues). If you do not run that backend, treat :shell-x paths as unsupported.

Caution

Review the _proxy() code before exposing it. It forwards method, headers (minus Host), data, and cookies to a constructed URL. Leaving it reachable without the intended backend produces errors; misconfiguring the target could create an SSRF-style forwarder. Disable or guard it for public deployments.

Hardening checklist

  • [ ] Terminate TLS at a reverse proxy; set X-Forwarded-For from the real peer.
  • [ ] Run bin/srv.py (gevent), never the Flask dev server, in production.
  • [ ] Run the container as a non-root user where possible (the stock Dockerfile fetches into /root/.cheat.sh).
  • [ ] Pin and regularly rebuild the image (Alpine 3.14 and redis:4-alpine are old — see §17).
  • [ ] Disable or firewall the :shell-x proxy unless intentionally used.
  • [ ] Monitor log/queries.log for abuse patterns.

12. Repository Structure

cheat.sh/
├── bin/
│   ├── app.py            Flask application (routes, UA detection, limits, POST)
│   ├── srv.py            gevent production server (entry point)
│   ├── clean_cache.py    delete specific Redis keys
│   └── release.py        release helper
├── lib/
│   ├── cheat_wrapper.py  query parse → route → postprocess → visualize
│   ├── routing.py        Router + routing table
│   ├── config.py         layered configuration (CONFIG dict)
│   ├── cache.py          Redis / no-op cache
│   ├── fetch.py          clone/update upstream repos + cache invalidation
│   ├── limits.py         per-IP rate limiting
│   ├── options.py        request-option parsing
│   ├── post.py           cheat-sheet submission (spool)
│   ├── postprocessing.py comment-adding + keyword filtering
│   ├── search.py         keyword search across topics
│   ├── standalone.py     offline CLI entry point
│   ├── stateful_queries.py  per-client last-query storage
│   ├── languages_data.py    language aliases, lexers, editor filetype maps
│   ├── globals.py        fatal/error/log helpers
│   ├── buttons.py        GitHub/Twitter HTML buttons
│   ├── adapter/          all source adapters (see §4)
│   ├── fmt/              comments / internal / markdown formatting
│   ├── frontend/         ansi.py, html.py renderers
│   └── panela/           internal coloring library
├── share/
│   ├── cht.sh.txt        the cht.sh client (served at /:cht.sh)
│   ├── ansi2html.sh      ANSI→HTML converter (used by frontend/html.py)
│   ├── bash_completion.txt, zsh.txt, fish.txt   shell completions
│   ├── vim.txt, emacs.txt, emacs-ivy.txt        editor helpers
│   ├── help.txt, intro.txt, firstpage*.txt      internal pages
│   ├── adapters/         chmod.sh, oeis.sh, rfc.sh (+ chmod.grc)
│   └── static/           CSS, images, opensearch.xml, malformed-response.html
├── etc/config.yaml       example config (note server.address quirk)
├── tests/                run-tests.sh, tests.txt, results/, unit tests
├── doc/                  standalone.md, README-ja.md
├── Dockerfile
├── docker-compose.yml
├── docker-compose.debug.yml
└── requirements.txt

Note

Internal pages such as :help, :intro, :bash_completion, and :vim are served verbatim from the corresponding share/*.txt files by the InternalPages adapter. To change :help, edit share/help.txt.


13. Module Reference

bin/app.py

The Flask application. Routes:

Route Methods Purpose
/ and /<path:topic> GET, POST Main query handler (answer)
/files/<path:path> GET Serve static assets from path.internal.static
/favicon.ico GET Serve favicon
/malformed-response.html GET Serve the malformed-response page

Helpers: _is_html_needed(), get_request_ip(), get_answer_language(), log_query(), is_result_a_script(), _proxy().

lib/cheat_wrapper.py

cheat_wrapper(query, request_options, output_format) — the single function the app calls. Normalizes the query (sanitize, add section name, rewrite aliases and editor section names), parses ~ searches, dispatches to routing.get_answers() or search.find_answers_by_keyword(), postprocesses, and visualizes.

lib/routing.py

Router builds the live adapter set from adapters.active ∪ adapters.mandatory, assembles the routing table, and exposes get_answers(), get_topic_type(), get_topics_list(), and :random handling. Topic-type results are memoized.

lib/config.py

Builds CONFIG (a Config(dict) subclass that absolutizes path.* keys against path.workdir). Loads defaults, then project file, then work-dir file, then env vars. Includes a doctested _get_nested() for hierarchical YAML keys.

lib/fetch.py

fetch_all() / update_all() / update_by_name() plus a CLI (fetch-all, update-all, update NAME). Detects duplicate repository locations, creates parent dirs, runs adapter-provided git commands, and invalidates cache entries for changed files.

lib/frontend/ansi.py

visualize() renders answer dicts to ANSI. _colorize_ansi_answer() selects a Pygments lexer by language (languages_data.LEXER), applies a 256-color terminal formatter, and optionally highlights only code blocks. remove_ansi() strips color when no-terminal (?T) is set.

lib/frontend/html.py

visualize() first renders ANSI, then pipes it through share/ansi2html.sh (bash + gawk + sed) to produce HTML, injecting a search form, topic datalist, edit button (for editable cheat.sheets topics), and GitHub/Twitter buttons.

lib/languages_data.py

The canonical data tables: LEXER (language → Pygments lexer), LANGUAGE_ALIAS (e.g. c++→cpp), VIM_NAME/SO_NAME/EMACS_FT_NAME/… (editor and StackOverflow name mappings), and rewrite_editor_section_name() / get_lexer_name().

lib/standalone.py

Offline entry point: lib/standalone.py [OPTIONS] QUERY. Forces cache.type=none, parses the query like a URL, and calls cheat_wrapper() directly — no server, no Redis. This is what the test harness uses in standalone mode.


14. Code Patterns & Conventions

  • Adapter polymorphism via subclass discovery. New sources require no registry edit beyond config — all_adapters() finds every Adapter subclass at runtime. This is the project's defining extensibility pattern.
  • Configuration as a flat dotted-key dict. CONFIG["cache.redis.host"] rather than nested access; Config.__setitem__ transparently absolutizes path keys.
  • Answer dicts as the internal contract. Adapters return {topic, topic_type, format, cache, answer[, filetype]}; frontends consume that shape. format is one of code, text, text+code, ansi.
  • ANSI-first rendering, HTML as a transform. HTML is produced by converting ANSI output, so terminal and browser views stay consistent.
  • Graceful degradation. Missing upstream binary → proxy to public server; connection failure → "Are you offline?" banner; unknown topic → fuzzy suggestions instead of a hard error.
  • Python 2/3 compatibility shims remain in several modules (from __future__ import print_function, six.with_metaclass, urlparse fallbacks) though the project targets Python 3.

Note

lib/search.py is explicitly described in its own docstring as "very naive… just a placeholder" — it iterates topics and pattern-matches. It is correct but not optimized; large recursive searches are bounded by search.limit.


15. Testing

The test suite lives in tests/:

# Unit tests (pytest)
python3 -m pytest -v lib/

# Input/output tests (golden-file comparison)
cd tests && ./run-tests.sh

run-tests.sh reads tests/tests.txt (one query per line) and diffs the output against recorded golden files in tests/results/. It can run in three modes:

  • standalone (default) — invokes lib/standalone.py (no server needed).
  • client — runs queries through share/cht.sh.txt against a live server.
  • curl — hits a running server at CHTSH_URL (default http://localhost:8002).

Useful environment toggles for the harness:

Variable Default Effect
CHEATSH_TEST_STANDALONE YES Use standalone mode
CHEATSH_TEST_SKIP_ONLINE NO Skip tests tagged [online]
CHEATSH_UPDATE_TESTS_RESULTS NO Re-record golden files instead of failing
CHTSH_URL http://localhost:8002 Server for client/curl modes

To run a server for the curl/client modes:

CHEATSH_CACHE_TYPE=none CHEATSH_PORT=50000 python3 bin/srv.py   # no caching (recommended)

There is also a focused unit test, lib/cheat_wrapper_test.py, that verifies the _add_section_name() query-rewriting logic (e.g. g++ is not split, but python copy file becomes python/copy file).


16. Troubleshooting (Operator)

500 Internal Server Error on normal queries

Cause: An adapter raised, an upstream timed out, or the :shell-x proxy was hit without its backend. Solution: Check log/main.log and container stderr; identify the failing adapter; ensure upstream repos are fetched (lib/fetch.py fetch-all) and Redis is reachable. Prevention: Health-check Redis and the upstream binary; disable :shell-x if unused.

docker-compose up fails to build or start

Cause: Native build failures (ICU/polyglot), or an outdated base image pulling incompatible package versions (e.g. a werkzeug/flask mismatch surfacing as cannot import name '_plain_int' from 'werkzeug._internal'). Solution: Pin compatible Flask/werkzeug versions; rebuild with --no-cache. Prevention: Modernize the base image and pin dependency versions (see §17).

Answers never update

Cause: Cache entries aren't being invalidated; upstream repos aren't being pulled. Solution: Run python3 lib/fetch.py update-all; verify .cached_revision files exist in each upstream/<repo> checkout. Prevention: Cron update-all regularly.

Rate limiting blocks legitimate traffic (or fails to block abuse)

Cause: X-Forwarded-For is missing (everyone shares one apparent IP) or spoofable (no trusted proxy). Solution: Put a reverse proxy in front that sets X-Forwarded-For from the real connection. Prevention: Never expose bin/srv.py directly.

HTML output is empty or errors

Cause: ansi2html.sh needs bash and gawk; if missing, HTML rendering fails. Solution: Ensure bash and gawk are installed in the runtime image (the Dockerfile installs them in the final stage). Prevention: Keep the HTML toolchain in your image.

:last returns an error

Cause: :last requires a session-id cookie. Solution: Have clients set an id cookie (the cht.sh client does this via its id command). This is expected behavior, not a fault.


17. Potential Improvements

Source-derived observations for maintainers and operators:

  • Modernize the container base. Dockerfile pins Alpine 3.14 and docker-compose.yml uses redis:4-alpine — both are old. Upgrading reduces CVE exposure and avoids the Flask/werkzeug import incompatibilities reported in issues.
  • Pin Python dependency versions. requirements.txt is largely unpinned (only colored<1.4.3). Pinning would prevent the werkzeug._internal._plain_int class of breakages.
  • Fix the etc/config.yaml example. It uses server.address, which is not a real config key; it should be server.bind to actually take effect.
  • Reconcile the port env var. bin/srv.py reads CHEATSH_PORT while the config system expects CHEATSH_SERVER_PORT. Documenting or unifying these would avoid confusion.
  • Replace the placeholder search. lib/search.py self-describes as naive; an indexed/ranked search would improve ~keyword quality and remove the hard search.limit cliff.
  • Harden or remove :shell-x. The _proxy() path is experimental, a frequent 500 source, and a potential SSRF vector if misconfigured. Gate it behind config or remove it from default builds.
  • Make rate-limit state shareable. Limits keeps counters in process memory, so multiple gevent workers/instances don't share limits. Backing them with Redis would make limiting consistent across a scaled-out deployment.
  • Document CHEATSH_CACHE_REDIS_PREFIX. The test script's CHEATSH_REDIS_PREFIX is a no-op; correcting it prevents accidental cache-namespace collisions in CI.
  • Run the container as non-root. Cheat sheets are fetched into /root/.cheat.sh; a dedicated unprivileged user would tighten the security posture.

Known issues (community-reported)

Issue Summary Operator note
#423, #381, #383, #422 500 Internal Server Error Often :shell-x or upstream timeouts
#406, #427 docker-compose up import/build errors werkzeug/flask version drift
#390 pip install on Windows fails Client is a shell script, not a pip package
#430 Install hangs on macOS Network stall / missing rlwrap
#403 History file location via env var (feature) Client enhancement
#433, #432, #425 Light-theme readability / source content UI + upstream content

18. Quick Reference

Entry points

Command Purpose
python3 bin/srv.py Production gevent server (server.bind:server.port)
python3 bin/srv.py --debug Enable Flask debug (limited under gevent)
CHEATSH_PORT=N python3 bin/srv.py Override the listen port
python3 lib/standalone.py QUERY Offline, server-less query
python3 lib/fetch.py fetch-all Clone all upstream cheat-sheet repos
python3 lib/fetch.py update-all Update repos + invalidate changed cache
python3 bin/clean_cache.py KEY... Delete specific Redis keys

Key environment variables

Variable Effect
CHEATSH_PORT Listen port (read directly by srv.py, highest precedence)
CHEATSH_CACHE_TYPE redis or none
CHEATSH_CACHE_REDIS_HOST / _PORT / _DB / _PREFIX Redis connection
CHEATSH_PATH_WORKDIR Override the work dir (default $HOME/.cheat.sh)
CHEATSH_CONFIG Override the config-file path
CHEATSH_UPSTREAM_URL / _TIMEOUT Fallback/proxy server
CHEATSH_SEARCH_LIMIT Max search results

Default ports & paths

Item Value
Listen 0.0.0.0:8002
Redis localhost:6379, db 0
Work dir $HOME/.cheat.sh
Upstream repos <workdir>/upstream/
Spool (POSTed sheets) <workdir>/spool/
Logs <workdir>/log/{main,queries,fetch}.log

Active adapters (default)

tldr, cheat, fosdem, translation, rosetta, late.nz, question, cheat.sheets, cheat.sheets dir, learnxiny, rfc, oeis, chmod (+ mandatory search, + always-on internal/unknown).