A complete, source-based reference for operators, developers, and architects who want to run, extend, or understand cheat.sh — the community-driven cheat sheet aggregation service.
This guide covers the system architecture, every configuration parameter, the deployment model, security posture, operational procedures, and the internals needed to modify or self-host the service. For end-user query syntax, see the cheat.sh User Guide.
Table of Contents
- Executive Summary
- Architecture
- Request Lifecycle
- The Adapter System
- Routing Engine
- System Requirements
- Configuration Reference
- Deployment
- Operations
- Caching
- Security
- Repository Structure
- Module Reference
- Code Patterns & Conventions
- Testing
- Troubleshooting (Operator)
- Potential Improvements
- Quick Reference
1. Executive Summary
cheat.sh is a Python web service that presents a single curl/browser/editor interface to many community cheat sheet repositories and StackOverflow. Its core design idea is that the URL path is the query and the response is plain, pasteable text (or, for browsers, an HTML rendering of that same text).
Key technical characteristics:
- Stateless HTTP service built on Flask, served in production by a gevent WSGI server for high concurrency.
- Pluggable adapter architecture — each upstream source (tldr-pages, the
cheatproject, Learn X in Y Minutes, RosettaCode, StackOverflow, plus computed sources like RFC/OEIS/chmod) is a subclass of a commonAdapterbase. - Redis-backed caching keyed by
topic_type:topic, with adapter-level control over what is cacheable. - Content-negotiated rendering — terminal user agents get ANSI text, browsers
get HTML produced by piping the ANSI output through an
ansi2htmlshell filter. - Self-hostable via Docker Compose (app container + Redis), with all cheat sheets fetched at image-build time.
Note
There is no database, no user accounts, and no authentication. State is limited to the Redis cache and optional per-client "last query" / session-id cookies. This keeps the service simple and horizontally scalable.
2. Architecture
High-level topology
┌──────────────────────────────────────────┐
curl / wget / xh ───► │ gevent WSGIServer (bin/srv.py) │
browser ───► │ ── wraps ── │
│ Flask app (bin/app.py) │
│ • User-Agent → ansi | html │
│ • rate limiting (lib/limits.py) │
│ • POST = cheat-sheet submission │
└───────────────┬──────────────────────────┘
│ cheat_wrapper(query, opts, fmt)
▼
┌──────────────────────────────────────────┐
│ cheat_wrapper (lib/cheat_wrapper.py) │
│ parse query → route → postprocess │
└───────────────┬──────────────────────────┘
│ get_answers(topic)
▼
┌──────────────────────────────────────────┐
│ Router (lib/routing.py) │
│ routing table → topic_type(s) │
│ cache lookup (lib/cache.py → Redis) │
└───────────────┬──────────────────────────┘
│ adapter.get_page_dict(topic)
▼
┌───────────────────────────────────────────────────────────────────┐
│ Adapters (lib/adapter/*) │
│ cheat.sheets · cheat · tldr · learnxiny · rosetta · question │
│ late.nz · fosdem · translation · rfc · oeis · chmod · internal │
└───────────────┬───────────────────────────────────────────────────┘
│ reads local git checkouts (upstream/) or runs commands
▼
┌───────────────────────────────────────────────────────────────────┐
│ Local cheat-sheet repositories (fetched by lib/fetch.py) │
│ + external upstream binary for StackOverflow questions │
└───────────────────────────────────────────────────────────────────┘
│
┌───────────────▼──────────────────────────┐
│ Frontends (lib/frontend/*) │
│ ansi.py → Pygments-highlighted ANSI │
│ html.py → ansi2html.sh → HTML page │
└──────────────────────────────────────────┘
Component inventory
| Component | File(s) | Responsibility |
|---|---|---|
| Production server | bin/srv.py |
gevent WSGI server, monkey-patches stdlib, binds host:port |
| Web application | bin/app.py |
Flask routes, User-Agent detection, rate limiting, POST handling, logging |
| Wrapper | lib/cheat_wrapper.py |
Parse/normalize the query, dispatch to router and search, postprocess |
| Router | lib/routing.py |
Map a topic to one or more adapters via the routing table; manage cache |
| Adapters | lib/adapter/*.py |
Retrieve answers from each source |
| Frontends | lib/frontend/*.py |
Render answer data as ANSI or HTML |
| Formatting | lib/fmt/*.py |
Comment beautification, internal-page colorization, markdown |
| Coloring lib | lib/panela/*.py |
Internal ANSI/color manipulation library (panela) |
| Config | lib/config.py |
Layered configuration (defaults → files → env) |
| Cache | lib/cache.py |
Redis (or no-op) cache |
| Rate limiting | lib/limits.py |
Per-IP query caps |
| Fetch/update | lib/fetch.py |
Clone and update upstream cheat-sheet repos; invalidate cache |
| Standalone | lib/standalone.py |
Offline command-line entry point (no server, no cache) |
| Client | share/cht.sh.txt |
The cht.sh shell client served at /:cht.sh |
Note
The service code lives in
lib/andbin/. Theshare/directory holds the client script, shell completions, editor helpers, static assets, and theansi2html.shconverter used for HTML rendering.
3. Request Lifecycle
A GET /<topic> request flows through the following steps (bin/app.py:answer):
- User-Agent inspection.
_is_html_needed()checks the User-Agent againstPLAIN_TEXT_AGENTS(curl,wget,httpie,lwp-request,python-requests,openbsd ftp,powershell,fetch,aiohttp,xh). A match means ANSI output; anything else (a browser) means HTML. - Option parsing.
options.parse_args(request.args)turns query-string flags into an options dict (see §7 → request options). - Special early exits. Favicon/apple-touch icons return empty;
:lastreturns the caller's previous query (requires a session-id cookie); POST requests are routed toprocess_post_request()(cheat-sheet submission). - Language negotiation.
get_answer_language()derives a human language from the*.cheat.shsubdomain, a?lang=argument, or theAccept-Languageheader. - Rate limiting. If the topic contains
+(a multi-word query),Limits.check_ip()is consulted; over-limit requests get429. - Dispatch.
cheat_wrapper(topic, request_options, output_format)runs the query and returns(result, found). - Rendering & logging. The result is returned as
text/plain(ANSI clients) or HTML (browsers). Every query is appended to the query log.
Inside cheat_wrapper() (lib/cheat_wrapper.py):
- Sanitize the query (strip
< > "). - Add section name — turn the first space/
+into a/sopython lambdabecomespython/lambda(with care forg++and friends). - Rewrite aliases & editor section names (
emacs:go-mode→go,c++→cpp). - Parse for a
~keywordsearch and search options. - Resolve —
find_answers_by_keyword()for searches, otherwiseget_answers()(the router). - Postprocess each answer (add language-appropriate comments, filter by keyword).
- Visualize via the chosen frontend (
ansi,html, orjson).
Important
The
jsonoutput format exists incheat_wrapper()butbin/app.pyonly ever requestshtmloransi. There is no JSON HTTP endpoint. Treat answers as text or HTML only.
4. The Adapter System
Every source of answers is an adapter — a subclass of Adapter
(lib/adapter/adapter.py). This is the central extensibility mechanism.
Base class contract
| Method | Purpose |
|---|---|
_get_list(prefix=None) |
Return the list of topics this adapter serves |
_get_page(topic, request_options) |
Return the answer text (or dict) for a topic |
is_found(topic) |
Whether this adapter has the topic (root only, by default) |
is_cache_needed() |
Whether answers should be cached (_cache_needed) |
get_page_dict(topic, ...) |
Wraps _get_page into the standard answer dict |
local_repository_location() |
Where the adapter's git checkout lives |
fetch_command() / update_command() |
Shell commands to clone/update the source |
Adapter discovery is automatic: all_adapters() walks Adapter.__subclasses__()
recursively, so adding a new adapter is as simple as defining a subclass in
lib/adapter/ and adding its name to adapters.active in config.
Adapter taxonomy
Adapter (abstract)
├── RepositoryAdapter file-backed cheat sheets
│ └── GitRepositoryAdapter adds git clone/pull/state for cache invalidation
│ ├── CheatSheets chubin/cheat.sheets (native sheets)
│ │ └── CheatSheetsDir directory listings (LANG/)
│ ├── Cheat cheat/cheatsheets
│ ├── Tldr tldr-pages/tldr (.md, multi-platform)
│ ├── LearnXinY adambard/learnxinyminutes (:learn pages)
│ ├── Rosetta acmeism/RosettaCodeData
│ └── Latenz chubin/late.nz (curlable latencies)
├── CommandAdapter answer = stdout of a shell command
│ ├── Fosdem current FOSDEM slide (demo)
│ ├── Translation external get_translation binary
│ ├── AdapterRfc share/adapters/rfc.sh
│ ├── AdapterOeis share/adapters/oeis.sh
│ └── AdapterChmod share/adapters/chmod.sh
├── UpstreamAdapter proxy to another cheat.sh server
│ └── Question StackOverflow via external upstream binary
├── InternalPages :help, :intro, :list, :styles, etc.
│ └── UnknownPages fuzzy "did you mean…" suggestions
└── Search marker adapter for the search route
Notable adapters
CheatSheets— the native sheets fromchubin/cheat.sheets, hierarchically organized; directory names are stored with a leading_that is stripped for queries.Tldr— reads tldr-pages.mdfiles, searchingcommon,linux,osx,sunos,windows,androidin order, and applies a trivial Markdown transform.LearnXinY— parses each language's*.html.markdownfile from Learn X in Y Minutes into blocks; powers the:learnpages. Each language has its own subclass with block-separator regexes.Question— answers free-form programming questions. It shells out to an external binary atpath.internal.bin.upstream; if that binary is absent, it falls back toUpstreamAdapter, proxying toupstream.url(defaulthttps://cheat.sh). It also runs language detection (polyglot) to support non-English questions.UpstreamAdapter— proxies to another cheat.sh instance; on connection failure it returns the "Are you offline?" ASCII banner and marks the answer non-cacheable.
Tip
When self-hosting without the proprietary StackOverflow upstream binary, the
questionadapter will transparently proxy to the publichttps://cheat.sh. Setupstream.urlto your own mirror, or accept the public default.
5. Routing Engine
Router (lib/routing.py) decides which adapter(s) answer a topic, using a
routing table assembled from three configurable lists plus a default:
routing_table = routing.pre + routing.main + routing.post
Each entry is (regexp, adapter_name). get_topic_type() walks the table; every
regex that matches and whose adapter reports is_found(topic) contributes a
route. If more than one matches, the trailing (default) route is dropped. If none
match, routing.default (question) is used.
Default routing table (from lib/config.py)
routing.pre (highest priority):
| Regex | Adapter | Matches |
|---|---|---|
^$ |
search |
Empty query |
^[^/]*/rosetta(/|$) |
rosetta |
…/rosetta paths |
^rfc/ |
rfc |
rfc/NNN |
^oeis/ |
oeis |
oeis/ANNN |
^chmod/ |
chmod |
chmod/NNN |
^: |
internal |
Special pages (:help, …) |
/:list$ |
internal |
Any …/:list |
/$ |
cheat.sheets dir |
Trailing slash → directory listing |
routing.main (the cheat-sheet sources, tried in order):
cheat.sheets, cheat, tldr, late.nz, fosdem, learnxiny.
routing.post:
| Regex | Adapter | Matches |
|---|---|---|
^[^/ +]*$ |
unknown |
A single bare word with no match → "did you mean" |
^[a-z][a-z]-[a-z][a-z]$ |
translation |
Locale pairs like en-de |
routing.default: question (StackOverflow).
Special routing behaviors
:random—handle_if_random_request()rewrites/LANG/:randomto a random real topic under that prefix before routing.- Explicit type — a topic of the form
type:topic(e.g.tldr:tar) forces a single adapter. questioncaching — question answers use a legacyq:<topic>cache key and are reused even across cache-format changes, because they are expensive to generate.
Note
The set of live adapters is
adapters.active ∪ adapters.mandatory. To disable a source, remove it fromadapters.active; to add one, add its name there (and ship the adapter class).
6. System Requirements
Runtime
| Requirement | Detail |
|---|---|
| Python | 3.x (the code carries some Python-2 compatibility shims but targets 3) |
| Redis | For caching (cache.type: redis); optional if cache.type: none |
git |
To fetch/update upstream cheat-sheet repositories |
bash + gawk + sed |
Required by ansi2html.sh for HTML rendering |
rlwrap |
Only on clients, for cht.sh --shell |
Python dependencies (requirements.txt)
| Package | Role |
|---|---|
gevent |
Async WSGI server (production) |
flask |
Web framework / routing |
requests |
Upstream proxying (Question/Upstream adapters) |
pygments |
Syntax highlighting + the list of color styles |
redis |
Cache backend |
fuzzywuzzy / (optional rapidfuzz) |
"Did you mean" fuzzy topic matching |
langdetect, polyglot, PyICU, pycld2 |
Human-language detection for questions |
colored, colorama |
Terminal color handling |
pyyaml |
Config + RosettaCode metadata parsing |
dateutils, cffi, python-Levenshtein |
Supporting libraries |
wheel, black, pytest |
Build / format / test tooling |
Warning
polyglot+PyICU+pycld2have native build dependencies (ICU, a C++ toolchain). The Dockerfile installs these via Alpine packages and a temporary build toolchain; expect non-trivial build steps if installing on bare metal.
7. Configuration Reference
Configuration is centralized in lib/config.py as a single CONFIG dictionary
with dotted keys. Paths are made absolute relative to path.workdir.
Resolution order (highest wins)
- Environment variables (
CHEATSH_*) - Config file in the work dir —
$WORKDIR/etc/config.yaml(or$CHEATSH_CONFIG) - Config file in the project dir —
<project>/etc/config.yaml - Built-in defaults in
_CONFIG
The work dir defaults to $HOME/.cheat.sh and can be overridden with
CHEATSH_PATH_WORKDIR.
Environment-variable mapping
A config key maps to an env var by: prefix with CHEATSH_, replace . with _,
uppercase. Only scalar (int/string) values can be set this way.
| Config key | Environment variable |
|---|---|
server.port |
CHEATSH_SERVER_PORT |
server.bind |
CHEATSH_SERVER_BIND |
cache.type |
CHEATSH_CACHE_TYPE |
cache.redis.host |
CHEATSH_CACHE_REDIS_HOST |
cache.redis.port |
CHEATSH_CACHE_REDIS_PORT |
cache.redis.db |
CHEATSH_CACHE_REDIS_DB |
cache.redis.prefix |
CHEATSH_CACHE_REDIS_PREFIX |
log.level |
CHEATSH_LOG_LEVEL |
upstream.url |
CHEATSH_UPSTREAM_URL |
upstream.timeout |
CHEATSH_UPSTREAM_TIMEOUT |
search.limit |
CHEATSH_SEARCH_LIMIT |
Caution
Two easy mistakes:
CHEATSH_PORTis special-cased inbin/srv.pyand read directly — it is not the same path as the config mapping. It takes precedence overserver.portwhen set.CHEATSH_SERVER_PORTworks too, via config.- The env var must match the full dotted key.
CHEATSH_REDIS_PREFIX(seen intests/run-tests.sh) does not map to anything — the correct variable isCHEATSH_CACHE_REDIS_PREFIX.
Full default configuration
| Key | Default | Meaning |
|---|---|---|
adapters.active |
tldr, cheat, fosdem, translation, rosetta, late.nz, question, cheat.sheets, cheat.sheets dir, learnxiny, rfc, oeis, chmod | Enabled sources |
adapters.mandatory |
["search"] |
Always-on adapters |
cache.type |
redis |
redis or none |
cache.redis.host |
localhost |
Redis host |
cache.redis.port |
6379 |
Redis port |
cache.redis.db |
0 |
Redis database number |
cache.redis.prefix |
"" |
Optional key prefix |
frontend.styles |
(all Pygments styles) | Allowed ?style= values |
log.level |
4 |
Verbosity (≥5 prints extra adapter debug) |
path.repositories |
upstream |
Where upstream repos are cloned |
path.spool |
spool |
Where POSTed cheat sheets are stored |
path.workdir |
$HOME/.cheat.sh |
Base work dir for all relative paths |
path.log.main |
log/main.log |
Application log |
path.log.queries |
log/queries.log |
Per-query log |
path.log.fetch |
log/fetch.log |
Fetch/update log |
path.internal.* |
(derived from project dir) | Static assets, pages, templates, ansi2html.sh, upstream binary |
routing.pre / routing.main / routing.post / routing.default |
(see §5) | Routing table |
upstream.url |
https://cheat.sh |
Fallback/proxy server |
upstream.timeout |
5 |
Upstream request timeout (seconds) |
search.limit |
20 |
Max answers returned for a search |
server.bind |
0.0.0.0 |
Bind address |
server.port |
8002 |
Listen port |
etc/config.yaml (shipped example)
server:
address: "0.0.0.0"
cache:
type: redis
Warning
The shipped
etc/config.yamluses the keyserver.address, but the real configuration key isserver.bind.server.addressdoes not map to any known key and is silently ignored (the default0.0.0.0is used regardless). When editing config, use the keys from the table above, not the example file's naming.
Request (per-query) options
Parsed by lib/options.py from the query string:
| Letter | Effect (internal flag) |
|---|---|
c |
add_comments=False, unindent_code=False |
C |
add_comments=False, unindent_code=True |
Q |
remove_text=True (code only) |
q |
quiet=True (suppress HTML chrome) |
T |
no-terminal=True (strip ANSI colors) |
Defaults: add_comments=True. Key-value args like style=native and lang=de
pass through directly.
8. Deployment
Docker Compose (recommended)
The simplest self-hosted setup uses the bundled docker-compose.yml:
docker-compose up
This builds the app image, fetches every cheat sheet into the image at build time, and starts two services:
services:
app:
build: .
image: cheat.sh
container_name: chtsh
depends_on:
- redis
environment:
- CHEATSH_CACHE_REDIS_HOST=redis
ports:
- "8002:8002"
volumes:
- .:/app:Z
redis:
image: redis:4-alpine
volumes:
- redis_data:/data
The service is then available at http://localhost:8002.
Important
The README explicitly notes the Docker setup is an early implementation, recommended for internal/dev/personal use rather than large public production.
The image build (Dockerfile)
Key build stages (Alpine 3.14 base):
- Install runtime packages:
git,py3-six,py3-pygments,py3-yaml,py3-gevent,libstdc++,py3-colorama,py3-requests,py3-icu,py3-redis,sed. - Copy the source into
/app. - Install remaining Python deps via a temporary build toolchain
(
py3-pip,g++,python3-dev,libffi-dev), then remove it. - Fetch all cheat sheets at build time:
python3 lib/fetch.py fetch-all(into/root/.cheat.sh/). - Install server packages:
py3-jinja2,py3-flask,bash,gawk. - Entry point:
python3 -u bin/srv.py.
Debug deployment
docker-compose.debug.yml overrides the app to run the Flask dev server (with
autoreload and tracebacks) instead of gevent:
docker-compose -f docker-compose.yml -f docker-compose.debug.yml up
It sets FLASK_ENV=development, FLASK_APP=bin/app.py, FLASK_RUN_HOST=0.0.0.0,
FLASK_RUN_PORT=8002.
Caution
Never expose the Flask dev server (
docker-compose.debug.yml) to the public internet. It is single-threaded, not hardened, and prints tracebacks. Usebin/srv.py(gevent) for anything reachable.
Running without Docker
# 1. Install dependencies
pip3 install -r requirements.txt
# 2. Fetch all upstream cheat sheets
python3 lib/fetch.py fetch-all
# 3. Start the production server (gevent)
python3 bin/srv.py # binds server.bind:server.port (0.0.0.0:8002)
CHEATSH_PORT=8080 python3 bin/srv.py # override the port
python3 bin/srv.py --debug # enable Flask debug (limited under gevent)
A reverse proxy (nginx, Traefik, …) typically terminates TLS in front of
bin/srv.py. The app reads X-Forwarded-For for the client IP, so configure your
proxy to set it.
9. Operations
Fetching and updating cheat sheets
All cheat-sheet content lives in cloned upstream repos under
path.repositories (upstream/ in the work dir). Manage them with lib/fetch.py:
python3 lib/fetch.py fetch-all # initial clone of every configured repo
python3 lib/fetch.py update-all # git pull each repo + invalidate changed cache entries
python3 lib/fetch.py update NAME # update one adapter's repo
update-all is the routine maintenance command: for each git-backed adapter it
pulls, computes the changed files since the last recorded state
(.cached_revision), and invalidates exactly those cache entries so updated
sheets are re-rendered while everything else stays warm.
Tip
Schedule
python3 lib/fetch.py update-all(e.g. via cron) to keep content fresh. Because it invalidates only changed entries, it is cheap to run often.
Logs
| Log | Path (relative to work dir) | Contents |
|---|---|---|
| Main | log/main.log |
Application/Flask logging (DEBUG level) |
| Queries | log/queries.log |
One line per query: IP found topic user_agent |
| Fetch | log/fetch.log |
Repository fetch/update activity |
Flask exceptions are additionally sent to stderr (with werkzeug duplicate-line
filtering), so container logs surface errors.
Cache maintenance
# delete specific Redis keys directly
python3 bin/clean_cache.py "tldr:tar" "cheat.sheets:python/lambda"
To flush everything, use redis-cli FLUSHDB against the configured database (be
aware this also drops session/last-query keys).
Submitting cheat sheets (spool)
POSTed cheat sheets (:post) are written to path.spool as
topic.name.<nonce> files for later review/merge. Monitor and drain this directory
as part of operations.
10. Caching
Caching is implemented in lib/cache.py and controlled per-adapter via
is_cache_needed().
Key scheme
| Key pattern | Used for |
|---|---|
<topic_type>:<topic> |
Standard cached answers (e.g. tldr:tar) |
q:<topic> |
StackOverflow question answers (legacy, expensive) |
l:<client_id> |
A client's last query (for :last) |
An optional global prefix (cache.redis.prefix) is prepended as <prefix>:.
Behavior
- Values that are
dict/listare JSON-serialized onputand parsed onget. - Adapters that set
_cache_needed = True(tldr, cheat, learnxiny, rosetta, translation, rfc, oeis, chmod, question) are cached; others (cheat.sheets, internal, search, upstream failures) are not. - An adapter's
_get_page()may override caching per-answer by returning a dict with a"cache"key (theUpstreamAdaptermarks offline answers"cache": False).
Note
Switching
cache.typetonone(as the standalone CLI and the test harness do) disables Redis entirely — every request is computed fresh. Useful for testing or a no-Redis deployment, at the cost of latency for expensive sources.
11. Security
cheat.sh has a deliberately small attack surface (no auth, no DB, no user data), but operators should understand the following.
Input handling
- Query sanitization —
cheat_wrapper._sanitize_query()strips< > "from the query before processing, reducing reflected-content risk in the HTML page. - Command adapters run shell programs.
CommandAdapter,AdapterRfc,AdapterOeis, andAdapterChmodexecute helper scripts viasubprocesswith argument lists (not shell strings), and the chmod/oeis adapters scrub their topic input with regexes ([^a-zA-Z0-9-]etc.) before passing it. This avoids shell injection through the topic. - The
fosdemandtranslationadapters invoke fixed binaries.fosdemuses asudoNOPASSWD entry for a single demo command, andtranslationreferences a hard-coded path (/home/igor/cheat.sh/bin/get_translation). These are environment-specific and inert on a default self-host.
Rate limiting
lib/limits.py caps queries per source IP for multi-word queries (those
containing +):
| Interval | Limit |
|---|---|
| Minute | 30 |
| Hour | 600 |
| Day | 1000 |
Over-limit requests receive 429 Not so fast!. There is a hard-coded whitelist
(_WHITELIST = ["5.9.243.177"]). The client IP is taken from X-Forwarded-For
(first value) when present, else remote_addr.
Warning
Because the IP comes from
X-Forwarded-For, only trust it behind a proxy you control. Ifbin/srv.pyis directly internet-facing, a client can spoof the header and bypass rate limits. Always front the service with a reverse proxy that setsX-Forwarded-Forfrom the real connection.
The :shell-x proxy
bin/app.py contains a _proxy() path for /:shell-x/ that forwards to a local
service on port 3000. This is a special/experimental feature and a known source
of 500 errors (see issues). If you do not run that backend, treat :shell-x
paths as unsupported.
Caution
Review the
_proxy()code before exposing it. It forwards method, headers (minusHost), data, and cookies to a constructed URL. Leaving it reachable without the intended backend produces errors; misconfiguring the target could create an SSRF-style forwarder. Disable or guard it for public deployments.
Hardening checklist
- [ ] Terminate TLS at a reverse proxy; set
X-Forwarded-Forfrom the real peer. - [ ] Run
bin/srv.py(gevent), never the Flask dev server, in production. - [ ] Run the container as a non-root user where possible (the stock Dockerfile
fetches into
/root/.cheat.sh). - [ ] Pin and regularly rebuild the image (Alpine 3.14 and
redis:4-alpineare old — see §17). - [ ] Disable or firewall the
:shell-xproxy unless intentionally used. - [ ] Monitor
log/queries.logfor abuse patterns.
12. Repository Structure
cheat.sh/
├── bin/
│ ├── app.py Flask application (routes, UA detection, limits, POST)
│ ├── srv.py gevent production server (entry point)
│ ├── clean_cache.py delete specific Redis keys
│ └── release.py release helper
├── lib/
│ ├── cheat_wrapper.py query parse → route → postprocess → visualize
│ ├── routing.py Router + routing table
│ ├── config.py layered configuration (CONFIG dict)
│ ├── cache.py Redis / no-op cache
│ ├── fetch.py clone/update upstream repos + cache invalidation
│ ├── limits.py per-IP rate limiting
│ ├── options.py request-option parsing
│ ├── post.py cheat-sheet submission (spool)
│ ├── postprocessing.py comment-adding + keyword filtering
│ ├── search.py keyword search across topics
│ ├── standalone.py offline CLI entry point
│ ├── stateful_queries.py per-client last-query storage
│ ├── languages_data.py language aliases, lexers, editor filetype maps
│ ├── globals.py fatal/error/log helpers
│ ├── buttons.py GitHub/Twitter HTML buttons
│ ├── adapter/ all source adapters (see §4)
│ ├── fmt/ comments / internal / markdown formatting
│ ├── frontend/ ansi.py, html.py renderers
│ └── panela/ internal coloring library
├── share/
│ ├── cht.sh.txt the cht.sh client (served at /:cht.sh)
│ ├── ansi2html.sh ANSI→HTML converter (used by frontend/html.py)
│ ├── bash_completion.txt, zsh.txt, fish.txt shell completions
│ ├── vim.txt, emacs.txt, emacs-ivy.txt editor helpers
│ ├── help.txt, intro.txt, firstpage*.txt internal pages
│ ├── adapters/ chmod.sh, oeis.sh, rfc.sh (+ chmod.grc)
│ └── static/ CSS, images, opensearch.xml, malformed-response.html
├── etc/config.yaml example config (note server.address quirk)
├── tests/ run-tests.sh, tests.txt, results/, unit tests
├── doc/ standalone.md, README-ja.md
├── Dockerfile
├── docker-compose.yml
├── docker-compose.debug.yml
└── requirements.txt
Note
Internal pages such as
:help,:intro,:bash_completion, and:vimare served verbatim from the correspondingshare/*.txtfiles by theInternalPagesadapter. To change:help, editshare/help.txt.
13. Module Reference
bin/app.py
The Flask application. Routes:
| Route | Methods | Purpose |
|---|---|---|
/ and /<path:topic> |
GET, POST | Main query handler (answer) |
/files/<path:path> |
GET | Serve static assets from path.internal.static |
/favicon.ico |
GET | Serve favicon |
/malformed-response.html |
GET | Serve the malformed-response page |
Helpers: _is_html_needed(), get_request_ip(), get_answer_language(),
log_query(), is_result_a_script(), _proxy().
lib/cheat_wrapper.py
cheat_wrapper(query, request_options, output_format) — the single function the
app calls. Normalizes the query (sanitize, add section name, rewrite aliases and
editor section names), parses ~ searches, dispatches to routing.get_answers()
or search.find_answers_by_keyword(), postprocesses, and visualizes.
lib/routing.py
Router builds the live adapter set from adapters.active ∪ adapters.mandatory,
assembles the routing table, and exposes get_answers(), get_topic_type(),
get_topics_list(), and :random handling. Topic-type results are memoized.
lib/config.py
Builds CONFIG (a Config(dict) subclass that absolutizes path.* keys against
path.workdir). Loads defaults, then project file, then work-dir file, then env
vars. Includes a doctested _get_nested() for hierarchical YAML keys.
lib/fetch.py
fetch_all() / update_all() / update_by_name() plus a CLI
(fetch-all, update-all, update NAME). Detects duplicate repository locations,
creates parent dirs, runs adapter-provided git commands, and invalidates cache
entries for changed files.
lib/frontend/ansi.py
visualize() renders answer dicts to ANSI. _colorize_ansi_answer() selects a
Pygments lexer by language (languages_data.LEXER), applies a 256-color terminal
formatter, and optionally highlights only code blocks. remove_ansi() strips color
when no-terminal (?T) is set.
lib/frontend/html.py
visualize() first renders ANSI, then pipes it through share/ansi2html.sh
(bash + gawk + sed) to produce HTML, injecting a search form, topic datalist,
edit button (for editable cheat.sheets topics), and GitHub/Twitter buttons.
lib/languages_data.py
The canonical data tables: LEXER (language → Pygments lexer), LANGUAGE_ALIAS
(e.g. c++→cpp), VIM_NAME/SO_NAME/EMACS_FT_NAME/… (editor and StackOverflow
name mappings), and rewrite_editor_section_name() / get_lexer_name().
lib/standalone.py
Offline entry point: lib/standalone.py [OPTIONS] QUERY. Forces cache.type=none,
parses the query like a URL, and calls cheat_wrapper() directly — no server, no
Redis. This is what the test harness uses in standalone mode.
14. Code Patterns & Conventions
- Adapter polymorphism via subclass discovery. New sources require no
registry edit beyond config —
all_adapters()finds everyAdaptersubclass at runtime. This is the project's defining extensibility pattern. - Configuration as a flat dotted-key dict.
CONFIG["cache.redis.host"]rather than nested access;Config.__setitem__transparently absolutizes path keys. - Answer dicts as the internal contract. Adapters return
{topic, topic_type, format, cache, answer[, filetype]}; frontends consume that shape.formatis one ofcode,text,text+code,ansi. - ANSI-first rendering, HTML as a transform. HTML is produced by converting ANSI output, so terminal and browser views stay consistent.
- Graceful degradation. Missing upstream binary → proxy to public server; connection failure → "Are you offline?" banner; unknown topic → fuzzy suggestions instead of a hard error.
- Python 2/3 compatibility shims remain in several modules
(
from __future__ import print_function,six.with_metaclass,urlparsefallbacks) though the project targets Python 3.
Note
lib/search.pyis explicitly described in its own docstring as "very naive… just a placeholder" — it iterates topics and pattern-matches. It is correct but not optimized; large recursive searches are bounded bysearch.limit.
15. Testing
The test suite lives in tests/:
# Unit tests (pytest)
python3 -m pytest -v lib/
# Input/output tests (golden-file comparison)
cd tests && ./run-tests.sh
run-tests.sh reads tests/tests.txt (one query per line) and diffs the output
against recorded golden files in tests/results/. It can run in three modes:
- standalone (default) — invokes
lib/standalone.py(no server needed). - client — runs queries through
share/cht.sh.txtagainst a live server. - curl — hits a running server at
CHTSH_URL(defaulthttp://localhost:8002).
Useful environment toggles for the harness:
| Variable | Default | Effect |
|---|---|---|
CHEATSH_TEST_STANDALONE |
YES |
Use standalone mode |
CHEATSH_TEST_SKIP_ONLINE |
NO |
Skip tests tagged [online] |
CHEATSH_UPDATE_TESTS_RESULTS |
NO |
Re-record golden files instead of failing |
CHTSH_URL |
http://localhost:8002 |
Server for client/curl modes |
To run a server for the curl/client modes:
CHEATSH_CACHE_TYPE=none CHEATSH_PORT=50000 python3 bin/srv.py # no caching (recommended)
There is also a focused unit test, lib/cheat_wrapper_test.py, that verifies the
_add_section_name() query-rewriting logic (e.g. g++ is not split, but
python copy file becomes python/copy file).
16. Troubleshooting (Operator)
500 Internal Server Error on normal queries
Cause: An adapter raised, an upstream timed out, or the :shell-x proxy was
hit without its backend. Solution: Check log/main.log and container stderr;
identify the failing adapter; ensure upstream repos are fetched
(lib/fetch.py fetch-all) and Redis is reachable. Prevention: Health-check
Redis and the upstream binary; disable :shell-x if unused.
docker-compose up fails to build or start
Cause: Native build failures (ICU/polyglot), or an outdated base image
pulling incompatible package versions (e.g. a werkzeug/flask mismatch surfacing as
cannot import name '_plain_int' from 'werkzeug._internal'). Solution: Pin
compatible Flask/werkzeug versions; rebuild with --no-cache. Prevention:
Modernize the base image and pin dependency versions (see §17).
Answers never update
Cause: Cache entries aren't being invalidated; upstream repos aren't being
pulled. Solution: Run python3 lib/fetch.py update-all; verify
.cached_revision files exist in each upstream/<repo> checkout. Prevention:
Cron update-all regularly.
Rate limiting blocks legitimate traffic (or fails to block abuse)
Cause: X-Forwarded-For is missing (everyone shares one apparent IP) or
spoofable (no trusted proxy). Solution: Put a reverse proxy in front that sets
X-Forwarded-For from the real connection. Prevention: Never expose
bin/srv.py directly.
HTML output is empty or errors
Cause: ansi2html.sh needs bash and gawk; if missing, HTML rendering
fails. Solution: Ensure bash and gawk are installed in the runtime image
(the Dockerfile installs them in the final stage). Prevention: Keep the HTML
toolchain in your image.
:last returns an error
Cause: :last requires a session-id cookie. Solution: Have clients set an
id cookie (the cht.sh client does this via its id command). This is expected
behavior, not a fault.
17. Potential Improvements
Source-derived observations for maintainers and operators:
- Modernize the container base.
Dockerfilepins Alpine 3.14 anddocker-compose.ymlusesredis:4-alpine— both are old. Upgrading reduces CVE exposure and avoids the Flask/werkzeug import incompatibilities reported in issues. - Pin Python dependency versions.
requirements.txtis largely unpinned (onlycolored<1.4.3). Pinning would prevent thewerkzeug._internal._plain_intclass of breakages. - Fix the
etc/config.yamlexample. It usesserver.address, which is not a real config key; it should beserver.bindto actually take effect. - Reconcile the port env var.
bin/srv.pyreadsCHEATSH_PORTwhile the config system expectsCHEATSH_SERVER_PORT. Documenting or unifying these would avoid confusion. - Replace the placeholder search.
lib/search.pyself-describes as naive; an indexed/ranked search would improve~keywordquality and remove the hardsearch.limitcliff. - Harden or remove
:shell-x. The_proxy()path is experimental, a frequent500source, and a potential SSRF vector if misconfigured. Gate it behind config or remove it from default builds. - Make rate-limit state shareable.
Limitskeeps counters in process memory, so multiple gevent workers/instances don't share limits. Backing them with Redis would make limiting consistent across a scaled-out deployment. - Document
CHEATSH_CACHE_REDIS_PREFIX. The test script'sCHEATSH_REDIS_PREFIXis a no-op; correcting it prevents accidental cache-namespace collisions in CI. - Run the container as non-root. Cheat sheets are fetched into
/root/.cheat.sh; a dedicated unprivileged user would tighten the security posture.
Known issues (community-reported)
| Issue | Summary | Operator note |
|---|---|---|
| #423, #381, #383, #422 | 500 Internal Server Error |
Often :shell-x or upstream timeouts |
| #406, #427 | docker-compose up import/build errors |
werkzeug/flask version drift |
| #390 | pip install on Windows fails |
Client is a shell script, not a pip package |
| #430 | Install hangs on macOS | Network stall / missing rlwrap |
| #403 | History file location via env var (feature) | Client enhancement |
| #433, #432, #425 | Light-theme readability / source content | UI + upstream content |
18. Quick Reference
Entry points
| Command | Purpose |
|---|---|
python3 bin/srv.py |
Production gevent server (server.bind:server.port) |
python3 bin/srv.py --debug |
Enable Flask debug (limited under gevent) |
CHEATSH_PORT=N python3 bin/srv.py |
Override the listen port |
python3 lib/standalone.py QUERY |
Offline, server-less query |
python3 lib/fetch.py fetch-all |
Clone all upstream cheat-sheet repos |
python3 lib/fetch.py update-all |
Update repos + invalidate changed cache |
python3 bin/clean_cache.py KEY... |
Delete specific Redis keys |
Key environment variables
| Variable | Effect |
|---|---|
CHEATSH_PORT |
Listen port (read directly by srv.py, highest precedence) |
CHEATSH_CACHE_TYPE |
redis or none |
CHEATSH_CACHE_REDIS_HOST / _PORT / _DB / _PREFIX |
Redis connection |
CHEATSH_PATH_WORKDIR |
Override the work dir (default $HOME/.cheat.sh) |
CHEATSH_CONFIG |
Override the config-file path |
CHEATSH_UPSTREAM_URL / _TIMEOUT |
Fallback/proxy server |
CHEATSH_SEARCH_LIMIT |
Max search results |
Default ports & paths
| Item | Value |
|---|---|
| Listen | 0.0.0.0:8002 |
| Redis | localhost:6379, db 0 |
| Work dir | $HOME/.cheat.sh |
| Upstream repos | <workdir>/upstream/ |
| Spool (POSTed sheets) | <workdir>/spool/ |
| Logs | <workdir>/log/{main,queries,fetch}.log |
Active adapters (default)
tldr, cheat, fosdem, translation, rosetta, late.nz, question,
cheat.sheets, cheat.sheets dir, learnxiny, rfc, oeis, chmod
(+ mandatory search, + always-on internal/unknown).