fd Admin & Architecture Guide

This guide is for operators, packagers, integrators, and developers who need to understand how fd works internally — how it is built, how it traverses the filesystem, how it applies filters, how it executes commands, and how it can be extended or hardened. End-user task documentation lives in the companion User Guide; this document covers everything behind it.

fd is a single-binary command-line tool written in Rust (edition 2024, MSRV 1.90.0). It has no daemon, no network surface, no persistent state, and no configuration file. Its behavior is determined entirely by command-line arguments, ignore files on disk, and a handful of environment variables. That simplicity is the dominant fact of its operations story.

1. Executive Summary

The codebase is small and cohesive: ~23 source files, ~182 KB of Rust under src/, organized into a flat top-level module set plus three sub-modules (exec, filter, fmt). The heavy lifting of directory traversal and gitignore semantics is delegated to BurntSushi's ignore crate (the same engine that powers ripgrep); fd itself is, in essence, a well-tuned configuration and filtering layer over that crate plus a parallel output/exec pipeline.

2. Architecture

2.1 High-level data flow

                          ┌──────────────────────────────────────────────┐
 argv ── Opts::parse() ──▶│  main.rs : run()                              │
 (clap derive, src/cli.rs)│   set_working_dir → search_paths             │
                          │   build pattern regexes (smart case, glob)   │
                          │   construct_config()  ──▶  Config            │
                          └───────────────────────┬──────────────────────┘
                                                   │ walk::scan(paths, regexps, config)
                                                   ▼
                          ┌──────────────────────────────────────────────┐
                          │  walk.rs : WorkerState                        │
                          │                                               │
                          │  build_walker()  → ignore::WalkParallel       │
                          │                                               │
                          │  spawn_senders()        receive()             │
                          │  ┌───────────────┐     ┌──────────────────┐  │
                          │  │ N walker       │     │ receiver thread  │  │
                          │  │ threads:       │ tx  │ ReceiverBuffer   │  │
                          │  │  traverse +    ├────▶│  buffer ≤100ms   │  │
                          │  │  filter +      │ rx  │  then stream,    │  │
                          │  │  batch results │     │  sorted if fast  │  │
                          │  └───────────────┘     └────────┬─────────┘  │
                          │   crossbeam bounded channel      │           │
                          └──────────────────────────────────┼───────────┘
                                                              ▼
                                          stdout (print_entry)  OR
                                          exec::job / exec::batch  (--exec / -X)
                                                              ▼
                                                        ExitCode.exit()

2.2 The two-phase pipeline

Phase 1 — Setup (main.rs). run() parses arguments into the Opts struct, optionally generates shell completions and returns early, sets the working directory if -C was given, resolves the search paths, and turns the raw pattern(s) into regex strings. Glob patterns are translated to regex via globset; --exact/--fixed-strings are escaped and optionally anchored. construct_config() then folds all of Opts into a single immutable Config struct (src/config.rs) that the rest of the program reads. Finally the regex strings are compiled into regex::bytes::Regex objects (byte regexes, so non-UTF-8 filenames still match) and walk::scan is called.

Phase 2 — Search (walk.rs). WorkerState owns the patterns, the config, and two atomic flags (quit_flag, interrupt_flag). scan():

1. Builds an ignore::WalkParallel walker from the config (build_walker).
2. Installs a Ctrl-C handler (when printing with colors) that sets the quit flag on first press and exits immediately on the second.
3. Creates a bounded crossbeam channel sized 2 × threads.
4. In a thread::scope, spawns the receiver, then runs the parallel walker's sender closures, then joins.

2.3 The producer side: parallel walking and filtering

spawn_senders() hands ignore::WalkParallel::run a closure that executes on every walker thread for every entry. The closure is the heart of fd's filtering and applies checks in a deliberate, cheap-to-expensive order so that most entries are rejected before any stat()-style syscall:

1. Quit check — bail out if the quit flag is set (Ctrl-C / broken pipe).
2. --ignore-contain — if a directory contains the named marker file, skip it (CACHEDIR.TAG-style pruning).
3. Root skip — depth-0 root entries are never reported.
4. Broken symlinks — NotFound errors on a symlink are converted into a reportable broken-symlink entry rather than an error.
5. --min-depth — drop entries above the minimum depth.
6. Name / full-path match — apply every compiled regex to the filename (or the absolute path when --full-path is set). All patterns must match (--and semantics).
7. --extension — a case-insensitive regex set built from the requested extensions.
8. --type — file-type filtering via the FileTypes bitmask (src/filetypes.rs).
9. --owner (Unix) — uid/gid checks against the entry's metadata.
10. --size — only for regular files; every size constraint must be satisfied.
11. --changed-within / --changed-before — modification-time constraints.
12. Color precomputation — when printing with colors, the entry's style is computed on the walker thread (parallelizing the work).
13. Send — the surviving entry is pushed into a batch and sent on the channel.
14. --prune — if set, return WalkState::Skip so matching directories are not descended into.

Results are sent in batches (Batch/BatchSender) rather than one at a time, which amortizes channel and lock overhead. The default batch limit is 0x100 (256); when running --exec in non-batch mode with multiple threads the limit drops to 1 to spread work evenly across the executor threads.

2.4 The consumer side: buffering, sorting, streaming

receive() chooses one of three behaviors based on the config:

• --exec-batch — drain the whole channel and call exec::batch once.
• --exec — spawn a pool of threads executor threads, each running exec::job, consuming results as they arrive.
• Plain printing — run a ReceiverBuffer on a locked, buffered stdout.

The ReceiverBuffer implements fd's signature output behavior. It starts in Buffering mode with a deadline of max_buffer_time (default 100 ms). If the search finishes within that window, the buffered results are sorted and printed — so quick searches give deterministic, alphabetized output. If the deadline passes or the buffer exceeds MAX_BUFFER_LENGTH (1000 entries), it switches to Streaming mode and prints results as they arrive (unsorted) for responsiveness on large searches. --max-results short-circuits the loop; -q/--quiet returns HasResults(true) on the first match without printing anything.

Note

This buffer-then-stream design is why a fd search of a small tree appears sorted while a search of a huge tree streams in filesystem order. It is intentional, and tunable through the hidden --max-buffer-time option.

2.5 Command execution (src/exec)

--exec/--exec-batch strings are parsed into a CommandSet of CommandTemplates, each a list of FormatTemplate tokens (src/fmt). Placeholders ({}, {/}, {//}, {.}, {/.}) are substituted per result. Key behaviors:

• If no placeholder appears in a template, an implicit {} is appended.
• One-by-one mode (exec::job) generates and runs one command per result; output is buffered per-command when running multi-threaded so concurrent commands don't interleave, and streamed directly when single-threaded.
• Batch mode (exec::batch) builds command lines incrementally with CommandBuilder, using the argmax crate to respect the OS maximum command-line length — it flushes and re-runs the command when the next argument wouldn't fit, or when --batch-size is reached. Batch templates are validated to contain at most one placeholder and a fixed executable as the first argument.
• Exit codes from all executed commands are merged: any non-zero result makes fd exit 1.

2.6 Module reference

3. Requirements

Build requirements

Cargo features

Note

jemalloc is deliberately disabled on macOS due to a known jemalloc/Catalina interaction (see the comment referencing sharkdp/fd#498 in Cargo.toml and the matching cfg on the global allocator in main.rs). It is also disabled on Windows, Android, the BSDs, illumos, 32-bit musl, and riscv64. These two locations must be kept in sync.

Runtime requirements

• No runtime dependencies for normal searching.
• --list-details/-l requires ls on PATH (GNU ls preferred; required on Windows).
• --owner is Unix-only (compiled out elsewhere).

Release build profile

The release profile is tuned for a small, fast binary:

[profile.release]
lto = true            # link-time optimization
strip = true          # strip symbols
codegen-units = 1     # maximize optimization (slower compile)

4. Configuration Mastery

fd has no configuration file. Its full configuration surface is: command-line flags (see the User Guide §5 for the complete list), ignore files, and environment variables.

4.1 Ignore-file precedence

fd layers several ignore sources, wired up in WorkerState::build_walker:

Important

--exclude/-E is implemented as an ignore override and beats all other ignore logic. Use it to force-skip a path regardless of what any ignore file says.

4.2 Environment variables

4.3 Color and hyperlink decision logic

Colorized output is enabled only when all of these hold (in auto mode): ANSI color is supported (always true on Unix; on Windows requires successful ANSI enablement or TERM), NO_COLOR is not set, and stdout is an interactive terminal. --color always forces it on; --color never forces it off. Hyperlinks (--hyperlink) default to never; in auto mode they are emitted only when color is enabled.

4.4 Path-output options

• --path-separator overrides the separator in printed paths (and is validated to be a single byte on Windows).
• --strip-cwd-prefix controls whether the leading ./ is stripped. The default (auto) strips it unless --print0 or an exec command is in play, where a leading ./ protects paths that begin with - from being mistaken for options.

5. Security Analysis

fd is a read-oriented local tool with a small, well-contained attack surface. The notable considerations for operators:

5.1 Command execution is the primary risk surface

--exec/--exec-batch run arbitrary programs with matched paths as arguments. This is by design and is the only way fd spawns processes (aside from ls for --list-details).

• Arguments are passed directly to the executable via argmax::Command (a thin wrapper over std::process::Command); fd does not invoke a shell, so there is no shell interpolation or word-splitting of the matched paths. This avoids the classic find -exec shell-injection class of problems.
• Because no shell is used, shell built-ins (cd, export) are not runnable directly — this is a frequent source of "Command not found" reports (issue #1944), and is also a safety property.

Caution

If you build fd -x/-X command lines from untrusted input (e.g. a user-supplied directory or pattern), the matched paths are safe (no shell), but never interpolate untrusted text into the command template itself. Treat the command template as code.

5.2 Symlink and filesystem boundaries

• Symlinked directories are not followed by default; --follow opts in. This prevents accidental traversal into linked trees or symlink loops.
• --one-file-system (--mount/--xdev) keeps the walk within a single filesystem, useful when scanning a mount point without descending into network or pseudo filesystems.
• Broken symlinks are reported as entries, not silently dropped, and don't crash the walk.

5.3 Filesystem errors

Errors such as permission-denied or dead symlinks are suppressed by default to keep output clean; --show-errors surfaces them. Operationally, absence of errors does not mean absence of unreadable paths — enable --show-errors when completeness matters.

5.4 Output and non-UTF-8 paths

fd matches and prints paths as bytes where possible (byte regexes; raw byte output on Unix non-interactive stdout), so it handles non-UTF-8 filenames faithfully. The --format path uses lossy UTF-8 conversion for output. Terminal hyperlinks percent-encode all non-ASCII and control bytes (src/hyperlink.rs), avoiding terminal-escape injection from crafted filenames in the hyperlink URL.

5.5 No network, no persistence, no privileged operations

fd opens no sockets, writes no files, and requires no elevated privileges. It reads directory contents and file metadata only. There is no telemetry. The supply-chain surface is the Rust crate dependency set (see §8) — pin and audit via Cargo.lock, which is committed.

5.6 Reporting vulnerabilities

The repository ships a SECURITY.md; follow its disclosure process for security reports rather than filing public issues.

6. Operations

fd is not a service, so "operations" means deployment, integration, and tuning rather than start/stop/monitor.

6.1 Deployment checklist

• [ ] Choose a distribution method (prebuilt release binary, cargo install fd-find, cargo binstall, or OS package).
• [ ] On Debian/Ubuntu, account for the fdfind binary name (provide an alias or symlink).
• [ ] Place the binary on PATH; verify with fd --version.
• [ ] (Optional) Install shell completions via fd --gen-completions <shell>.
• [ ] (Optional) Create a global ignore file at <config-dir>/fd/ignore for machine-wide defaults.
• [ ] If --list-details will be used on Windows, ensure GNU ls is installed.

6.2 Performance tuning

Tip

For the fastest scans, combine ignore-based pruning with --max-depth and, when running commands, prefer -X (batch) over -x (per-result) to minimize process spawns.

6.3 Signal handling

When printing with colors, fd installs a Ctrl-C handler: the first Ctrl+C sets a quit flag for a clean shutdown; a second forces immediate exit with code 130. On Unix, fd re-raises SIGINT with the default handler before exiting so the parent shell sees a genuine interrupted-by-signal status.

6.4 Integration patterns

• Pipelines: use -0/--print0 with xargs -0 to handle paths containing spaces or newlines safely.
• Scripts/CI: use -q/--quiet for existence checks (exit 0 = found, 1 = not found).
• Editors/tools: fd is commonly used as the backend file lister for fuzzy finders and editors; --type f, --hidden, and ignore handling make it a good default source.

7. Code Quality & Conventions

• Separation of concerns: parsing (cli), configuration (config), traversal (walk), filtering (filter, filetypes), formatting (fmt, output), and execution (exec) are cleanly separated. Config is the single immutable hand-off between setup and search.
• Cheap-before-expensive filtering: the walker closure orders checks to avoid stat syscalls on entries that fail a name match — a deliberate performance discipline.
• Lazy work: DirEntry caches metadata and computed style in OnceCells so they're fetched at most once.
• Test coverage: unit tests are colocated with their modules — extensive table-driven tests for size parsing (filter/size.rs), time filtering (filter/time.rs, with an injectable clock), owner parsing, format-template generation, path helpers, exit-code merging, and Windows path-separator handling. Integration tests live in tests/ with a TestEnv harness (tests/testenv/mod.rs).
• Cross-platform care: #[cfg(...)] gates isolate Unix-only (owner, signals, raw byte output) and Windows-only (path separators, UNC handling, ANSI enablement) code paths.
• Formatting: rustfmt.toml and a Makefile standardize builds and style; CI badges and a release-checklist.md document the release process.

Note

src/output.rs carries TODO: this function is performance critical and can probably be optimized markers on the hot print paths — a known, intentional optimization target, not a defect.

8. Dependencies

Curated from Cargo.toml. Cargo.lock is committed, pinning the full transitive set.

9. Potential Improvements & Known Issues

Drawn from in-code comments and community-reported issues; useful for maintainers and downstream packagers.

Open community issues of operational interest

In-code improvement targets

• jemalloc on macOS — currently disabled pending an upstream fix (#498); revisit when the jemalloc/macOS interaction is resolved. Keep the Cargo.toml feature gate and the main.rs allocator cfg in sync.
• src/output.rs hot paths — flagged in-source as performance-critical and optimizable; the print functions are the most-executed code in large searches.
• Leading-dot detection (regex_helper.rs) — currently detects only the simplest ^\. form; more patterns that exclusively match hidden files could be recognized to improve the "hidden files are filtered" hint.

10. Quick Reference

Build & install

cargo build --release            # build (needs Rust ≥ 1.90.0)
cargo install fd-find            # install from crates.io (binary: fd)
cargo binstall fd-find           # install a prebuilt release binary

Architecture in one line: parse → Config → ignore::WalkParallel sender threads filter & batch → crossbeam channel → receiver buffers ≤100 ms (sorted) then streams, or fans out to exec threads → ExitCode.exit().

Exit codes: 0 success / match found · 1 general error / no match (--quiet) · 130 SIGINT.

Default behaviors: smart case · skip hidden · respect ignore files (in-repo) · match name only · regex pattern · color/ hyperlink in interactive terminals.

No config file. Configure via flags, ignore files (.fdignore, global <config>/fd/ignore, --ignore-file), and env vars (LS_COLORS, NO_COLOR, MSYSTEM, TERM).

Key source files: main.rs (orchestration/config), cli.rs (flags), walk.rs (engine), exec/ (commands), filter/ (size/time/owner), output.rs (printing).