This guide is the operational and architectural reference for Paperclip — a self-hosted platform that orchestrates teams of AI coding agents to do work autonomously. It covers the system architecture, every configurable subsystem, deployment topologies, security posture, the data model, the API surface, and the internals a developer would need to extend or operate the platform.
It is written against source commit 03e1e3a (package version 0.3.1, MIT). For end-user/operator tasks (hiring agents, filing issues, running the CLI day-to-day), see the Paperclip User Guide; this document assumes that material and goes deeper.
1. Executive Summary
Paperclip is a single-deployment control plane for an AI workforce. Operators stand up one server that hosts:
- a REST API (Express) with ~300+ endpoints and an auto-generated OpenAPI document,
- a React/Vite single-page UI served from the same process,
- a
paperclipaiCLI for setup, operations, and developer worktrees, - an embedded PostgreSQL (or an external one) accessed through Drizzle ORM,
- a plugin runtime (sandboxed JSON-RPC workers) and a plugin SDK,
- an MCP server so external LLM clients can drive the instance, and
- a fleet of interchangeable agent adapters and sandbox providers.
The mental model is a company: companies contain agents (backed by adapters) that work issues in environments/sandboxes, governed by budgets, approvals, routines, secrets, and an org chart.
| Attribute | Value |
|---|---|
| Language | TypeScript (ESM), Node ≥ 20 |
| Monorepo | pnpm workspaces (pnpm@9.15.4) |
| Server | Express + OpenAPI registry + Zod |
| Persistence | PostgreSQL via Drizzle ORM (embedded embedded-postgres, or external) |
| Auth | better-auth (sessions) + API keys + agent JWTs |
| UI | React 19 + Vite (+ Storybook, 40-locale i18n) |
| Default port | 3100 |
| Distribution | npm package paperclipai (CLI), plus publishable workspace packages |
2. Architecture
2.1 Component diagram
┌───────────────────────────────────────────────┐
Browser ──HTTP/WS────► │ Paperclip Server │
(React SPA, Vite) │ (Express) │
│ │
CLI ──────HTTP───────► │ middleware/ ─► routes/ ─► services/ │
(paperclipai) │ (auth, (40 files) (117 files: │
│ logging, agents, │
MCP client ─stdio────► │ redaction) issues, │
(paperclip-mcp-server) │ runs, │
│ plugins, │
│ costs, …) │
│ │ │ │ │
│ Drizzle ORM storage (local/S3) secrets │
│ │ │
└────────┼────────────────────────────────────────┘
▼
PostgreSQL (embedded or external; 86 schema files)
Agent execution (dispatched by services through adapters):
┌───────────────────────────── adapters/ ─────────────────────────────┐
│ claude-local · codex-local · cursor-local · cursor-cloud · │
│ gemini-local · grok-local · opencode-local · pi-local · │
│ acpx-local · openclaw-gateway (each: index/server/ui/cli) │
└──────────────────────────────────────────────────────────────────────┘
Execution surface (sandbox provider plugins + bridge):
┌──────────────── sandbox-providers/ ────────────────┐
│ e2b · daytona · modal · cloudflare · exe-dev · │
│ fake-sandbox ⇄ sandbox-callback bridge │
└─────────────────────────────────────────────────────┘
2.2 Monorepo layout
| Package | Path | Responsibility |
|---|---|---|
| CLI | cli/ |
paperclipai binary (Commander). Onboarding, run, doctor, configure, db backup, worktrees, heartbeat, routines |
| Server | server/ |
Express API: routes/ (40 files), services/ (117 files), middleware/, adapters/, storage/ |
| UI | ui/ |
React 19 + Vite SPA: pages/ (84), components/ (200), api/ (40), lib/ (162), i18n/ (40 locales) |
| Shared | packages/shared/ |
Zod validators, shared types, telemetry, document-anchors |
| DB | packages/db/ |
Drizzle schema (86 files), 94 migrations, embedded-postgres, backup/restore |
| Adapter Utils | packages/adapter-utils/ |
Execution targets, sandbox-callback bridge, SSH, server utils — the shared runtime adapters build on |
| Adapters | packages/adapters/* |
10 agent adapters (claude/codex/cursor-local/cursor-cloud/gemini/grok/opencode/pi/acpx/openclaw-gateway) |
| Plugin SDK | packages/plugins/sdk/ |
Plugin protocol (JSON-RPC), host-client factory, capability model, UI kit |
| Sandbox providers | packages/plugins/sandbox-providers/* |
e2b, daytona, modal, cloudflare (+bridge-template), exe-dev |
| Example/utility plugins | packages/plugins/* |
workspace-diff, llm-wiki, file-browser, hello-world, kitchen-sink, orchestration-smoke, fake-sandbox, create-paperclip-plugin |
| MCP server | packages/mcp-server/ |
paperclip-mcp-server bin (MCP SDK + Zod) |
| Skills catalog | packages/skills-catalog/ |
Bundled + optional agent skills, catalog builder |
All workspace packages publish under the @paperclipai/* scope; the CLI publishes as paperclipai. Internal dependencies use workspace:* so the CLI bundles the adapters, db, server, and shared packages it needs.
2.3 Request lifecycle
- A request reaches Express; middleware runs first — request logging, log redaction (
server/src/log-redaction.tsscrubs user/host identifiers), and authentication (middleware/auth.ts). - Auth resolves the actor — a board (human) user, an agent, or anonymous — from a session cookie, an API key, an agent JWT, or a cloud-tenant token.
- The matched route (
server/src/routes/*) validates input with Zod and delegates to one or more services (server/src/services/*). - Services read/write PostgreSQL via Drizzle, persist files via the storage layer, and resolve credentials via the secrets manager.
- Agent work is dispatched through the relevant adapter, which runs the underlying coding agent — locally or inside a sandbox via a provider plugin, mediated by the callback bridge.
- Plugins receive events and expose tools over the JSON-RPC plugin bridge.
2.4 The adapter contract
Every adapter package exports four entry points so the same adapter can plug into each layer of the stack:
| Export | Consumed by |
|---|---|
. (index) |
Core registration / capability metadata |
./server |
Server-side execution logic |
./ui |
UI surface (config form, model pickers) |
./cli |
CLI integration (e.g. quota probes) |
Adapters are registered in a server-side registry (server/src/adapters/registry.ts) and managed at runtime through /api/adapters — install, enable, reload, reinstall, and fetch a config schema. Unknown adapter references raise Unknown adapter type: <type>. Adapters can communicate over a child-process or HTTP transport (server/src/adapters/process, server/src/adapters/http).
Note
Adapters are hot-manageable:
POST /adapters/:type/reloadre-reads configuration andPOST /adapters/:type/reinstallperforms a full teardown/re-init, both without restarting the server.
3. System Requirements
| Component | Requirement |
|---|---|
| Node.js | ≥ 20 |
| Package manager (dev) | pnpm 9.15.4 |
| PostgreSQL | Embedded (bundled embedded-postgres, 18.x beta) by default; external Postgres for production |
| OS | Linux / macOS / Windows (WSL recommended) |
| TypeScript (dev) | 5.7.x |
| Build tooling | esbuild (CLI/plugins), Vite (UI), drizzle-kit (migrations), tsc |
Provider credentials
Agents and sandboxes need third-party credentials, supplied as environment variables or Paperclip secrets:
| Purpose | Variable(s) |
|---|---|
| Anthropic / Claude | ANTHROPIC_API_KEY |
| OpenAI / Codex | OPENAI_API_KEY |
| E2B sandbox | E2B_API_KEY |
| Daytona sandbox | DAYTONA_API_KEY |
| Modal sandbox | MODAL_TOKEN_ID, MODAL_TOKEN_SECRET |
| exe-dev sandbox | EXE_API_KEY |
4. Deployment Topologies
Paperclip is a single-server application; there is no built-in horizontal scaling (the embedded Postgres and bridge queue assume one process owns the state). Scale vertically and offload the database/storage for heavier loads.
4.1 Reachability and deployment posture
Two orthogonal axes control exposure. The default for a fresh instance is local_trusted + private.
| Axis | Env var | Values |
|---|---|---|
| Mode (auth) | PAPERCLIP_DEPLOYMENT_MODE |
local_trusted (loopback, no login) · authenticated (login required) |
| Exposure | PAPERCLIP_DEPLOYMENT_EXPOSURE |
private · public |
| Bind preset | PAPERCLIP_BIND / --bind |
loopback · lan · tailnet |
The server enforces the safe combinations at startup: local_trusted requires loopback host binding and private exposure; authenticated + public requires an explicit public base URL. Anything reachable off-host should therefore run authenticated.
Caution
Never run
local_trustedmode off loopback — it performs no authentication, and the server will refuse to start (local_trusted mode requires loopback host binding). The onboarding wizard switches toauthenticatedautomatically forlan/tailnetbinds; preserve that when scripting deployments.
4.2 Local / single user
npx paperclipai onboard --yes --run # local_trusted + loopback, embedded Postgres
Zero external dependencies — the embedded Postgres and local storage are provisioned automatically under ~/.paperclip.
4.3 Docker (quickstart)
docker/docker-compose.quickstart.yml builds the image and runs the server on port 3100 with a bind-mounted data volume:
BETTER_AUTH_SECRET="$(openssl rand -hex 32)" \
docker compose -f docker/docker-compose.quickstart.yml up --build
Key container environment (from the compose file):
| Var | Value |
|---|---|
HOST |
0.0.0.0 |
PAPERCLIP_HOME |
/paperclip |
PAPERCLIP_DEPLOYMENT_MODE |
authenticated |
PAPERCLIP_DEPLOYMENT_EXPOSURE |
private |
BETTER_AUTH_SECRET |
required (compose fails fast without it) |
4.4 AWS ECS Fargate
docker/ecs-task-definition.json is a ready template for a public, authenticated deployment:
- Fargate task: 2 vCPU / 4 GB, container port 3100.
- Environment:
NODE_ENV=production,SERVE_UI=true,PAPERCLIP_DEPLOYMENT_MODE=authenticated,PAPERCLIP_DEPLOYMENT_EXPOSURE=public,PAPERCLIP_MIGRATION_AUTO_APPLY=true,HEARTBEAT_SCHEDULER_ENABLED=true. - Secrets pulled from AWS Secrets Manager:
DATABASE_URL,BETTER_AUTH_SECRET,ANTHROPIC_API_KEY,OPENAI_API_KEY,GITHUB_TOKEN. - Persistent data on EFS (encrypted in transit) mounted at
/paperclip. - Health check:
curl -f http://localhost:3100/api/health. - Logs to CloudWatch (
/ecs/paperclip).
Important
For public deployments, point
PAPERCLIP_PUBLIC_URL(and, if you fix the auth base URL explicitly,PAPERCLIP_AUTH_PUBLIC_BASE_URLwithPAPERCLIP_AUTH_BASE_URL_MODE=explicit) at the externally reachable URL, and setBETTER_AUTH_TRUSTED_ORIGINSfor any additional front-end origins.
4.5 Podman / systemd (Quadlet)
docker/quadlet/ provides Quadlet unit files for running the container as a systemd-managed service (Podman). Use this for rootless, auto-restarting host deployments without a full container orchestrator.
4.6 Untrusted review sandbox
docker/docker-compose.untrusted-review.yml + docker/untrusted-review/Dockerfile run a hardened container for reviewing agent-produced changes: cap_drop: ALL, no-new-privileges, a size-limited tmpfs for /tmp, and isolated volumes. Use it when you want to inspect or execute agent output away from any trusted context.
4.7 External PostgreSQL
export DATABASE_URL="postgresql://user:pass@db-host:5432/paperclip"
export PAPERCLIP_MIGRATION_AUTO_APPLY=true
paperclipai run
Provide DATABASE_MIGRATION_URL separately if migrations need a more privileged connection than the runtime user. Embedded Postgres data does not auto-migrate to an external database — dump and restore it deliberately.
4.8 Reverse proxy
Terminate TLS at nginx/Traefik/Caddy and proxy to the server port (3100), forwarding Upgrade/Connection headers for WebSocket and using long read timeouts (agent runs are long-lived). Set PAPERCLIP_PUBLIC_URL and BETTER_AUTH_TRUSTED_ORIGINS to the external origin.
4.9 Startup validation errors
The server validates the deployment configuration on boot and exits with a precise message if it is unsafe or incomplete:
| Error | Meaning / fix |
|---|---|
local_trusted mode requires loopback host binding (received: <host>) |
You bound local_trusted to a non-loopback host. Bind loopback, or switch to authenticated. |
local_trusted mode only supports private exposure |
local_trusted + public is rejected. Use authenticated for public exposure. |
authenticated public exposure requires auth.baseUrlMode=explicit |
Set PAPERCLIP_AUTH_BASE_URL_MODE=explicit. |
auth.baseUrlMode=explicit requires auth.publicBaseUrl |
Set PAPERCLIP_AUTH_PUBLIC_BASE_URL to your public URL. |
reachable postgres does not use the expected embedded data directory |
Another Postgres occupies the embedded port. Stop it, change PAPERCLIP_EMBEDDED_POSTGRES_PORT, or set DATABASE_URL. |
Migrations are still pending after migration-history reconciliation |
Migration state couldn't auto-resolve. Back up, inspect the migrations history table, then resolve manually. |
Tip
Run
paperclipai doctor(with--repair) to surface and fix most of these before they block a start;paperclipai envprints the fully resolved deployment configuration so you can verify mode/exposure/host/port up front.
5. Configuration Reference
Configuration comes from environment variables plus a per-instance JSON config (PAPERCLIP_CONFIG, default under ~/.paperclip/instances/<id>/config.json). The onboard wizard writes the config; paperclipai configure -s <section> edits sections (llm, database, logging, server, storage, secrets); paperclipai env prints the resolved environment. Environment variables override the file; the file overrides built-in defaults.
The tables below catalog the environment variables Paperclip reads, grouped by subsystem. Defaults are shown only where confirmed in source; where a default is environment-derived or set during onboarding, it is left blank — run paperclipai env to see the resolved value for your instance.
5.1 Core & server
| Variable | Default | Purpose |
|---|---|---|
PAPERCLIP_HOME |
~/.paperclip |
Root data directory |
PAPERCLIP_INSTANCE_ID |
default |
Instance isolation key (validated; alphanumeric/dash) |
PAPERCLIP_CONFIG |
under PAPERCLIP_HOME |
Path to the instance JSON config |
PAPERCLIP_CONTEXT / profile |
— | CLI context file/profile selection |
HOST / PAPERCLIP_BIND_HOST / PAPERCLIP_LISTEN_HOST |
127.0.0.1 (in-process) |
Bind host |
PORT / PAPERCLIP_LISTEN_PORT |
3100 |
Listen port |
PAPERCLIP_BIND |
— | Bind preset: loopback/lan/tailnet |
PAPERCLIP_TAILNET_BIND_HOST |
— | Explicit tailnet bind host |
SERVE_UI |
— | Serve the compiled React UI from the server |
PAPERCLIP_UI_DEV_MIDDLEWARE |
— | Enable Vite dev middleware (HMR) |
PAPERCLIP_OPEN_ON_LISTEN |
— | Open a browser when the server starts |
PAPERCLIP_LOG_DIR |
— | Log directory |
NODE_ENV |
— | development / production / test |
5.2 Authentication & authorization
| Variable | Purpose |
|---|---|
PAPERCLIP_DEPLOYMENT_MODE |
local_trusted (default) or authenticated |
PAPERCLIP_DEPLOYMENT_EXPOSURE |
private (default) or public |
BETTER_AUTH_SECRET |
Required in authenticated mode. Signs sessions |
BETTER_AUTH_URL / BETTER_AUTH_BASE_URL |
better-auth URLs |
BETTER_AUTH_TRUSTED_ORIGINS |
Comma-separated trusted CORS origins |
PAPERCLIP_PUBLIC_URL |
Public-facing base URL |
PAPERCLIP_AUTH_PUBLIC_BASE_URL / PAPERCLIP_AUTH_BASE_URL_MODE |
Explicit auth base URL + derivation mode (auto/explicit) |
PAPERCLIP_ALLOWED_HOSTNAMES |
Comma-separated hostnames permitted to reach the server |
PAPERCLIP_AUTH_DISABLE_SIGN_UP |
Disable new account registration |
PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN |
Cloud-tenant server auth token |
5.3 Agent identity (JWT)
| Variable | Purpose |
|---|---|
PAPERCLIP_AGENT_JWT_SECRET |
Signing key for agent JWTs (falls back to the auth secret) |
PAPERCLIP_AGENT_JWT_TTL_SECONDS |
Token lifetime |
PAPERCLIP_AGENT_JWT_ISSUER |
JWT iss claim |
PAPERCLIP_AGENT_JWT_AUDIENCE |
JWT aud claim |
5.4 Database & backups
| Variable | Default | Purpose |
|---|---|---|
DATABASE_URL |
embedded | Postgres connection URL (set for external DB) |
DATABASE_MIGRATION_URL |
= DATABASE_URL |
Separate connection for migrations |
PAPERCLIP_EMBEDDED_POSTGRES_PORT |
— | Port for the embedded Postgres |
PAPERCLIP_EMBEDDED_POSTGRES_VERBOSE |
— | Verbose embedded-Postgres logging |
PAPERCLIP_MIGRATION_AUTO_APPLY |
— | Apply pending migrations on startup |
PAPERCLIP_MIGRATION_PROMPT |
— | Prompt before applying migrations |
PAPERCLIP_PG_DUMP_PATH / PAPERCLIP_PSQL_PATH |
pg_dump/psql |
Binary paths for backup/restore |
PAPERCLIP_DB_BACKUP_ENABLED |
— | Enable automated backups |
PAPERCLIP_DB_BACKUP_INTERVAL_MINUTES |
— | Backup cadence |
PAPERCLIP_DB_BACKUP_RETENTION_DAYS |
— | Backup retention window |
PAPERCLIP_DB_BACKUP_DIR |
— | Backup output directory |
5.5 Storage
| Variable | Purpose |
|---|---|
PAPERCLIP_STORAGE_PROVIDER |
local or s3 |
PAPERCLIP_STORAGE_LOCAL_DIR |
Local storage directory |
PAPERCLIP_STORAGE_S3_BUCKET / _REGION / _ENDPOINT / _PREFIX / _FORCE_PATH_STYLE |
S3 (or S3-compatible: R2, MinIO) configuration |
5.6 Secrets
| Variable | Purpose |
|---|---|
PAPERCLIP_SECRETS_PROVIDER |
local (encrypted file) or AWS-backed |
PAPERCLIP_SECRETS_MASTER_KEY / _FILE |
Master encryption key (inline or file) |
PAPERCLIP_SECRETS_STRICT_MODE |
Fail on any secret-decryption error |
PAPERCLIP_SECRETS_AWS_DEPLOYMENT_ID / _AWS_REGION / _AWS_KMS_KEY_ID |
AWS KMS envelope-encryption configuration |
| (standard AWS env) | AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AWS_PROFILE, AWS_ROLE_ARN, web-identity & container-credential vars are all honored by the AWS secrets path |
5.7 Scheduler, bridge, runtime & limits
| Variable | Default | Purpose |
|---|---|---|
HEARTBEAT_SCHEDULER_ENABLED |
— | Enable the agent heartbeat scheduler |
HEARTBEAT_SCHEDULER_INTERVAL_MS |
30000 |
Scheduler tick interval |
PAPERCLIP_BRIDGE_QUEUE_DIR |
— | Sandbox-callback bridge queue directory |
PAPERCLIP_BRIDGE_TOKEN |
— | Bridge authentication token |
PAPERCLIP_BRIDGE_HOST / _PORT |
— | Bridge listener address |
PAPERCLIP_BRIDGE_POLL_INTERVAL_MS |
— | Bridge queue poll interval |
PAPERCLIP_BRIDGE_RESPONSE_TIMEOUT_MS |
— | Bridge RPC timeout |
PAPERCLIP_BRIDGE_MAX_QUEUE_DEPTH / _MAX_BODY_BYTES |
— | Bridge backpressure & size limits |
PAPERCLIP_RUNTIME_API_URL / PAPERCLIP_API_URL / PAPERCLIP_RUNTIME_API_CANDIDATES_JSON |
— | Agent↔server runtime API discovery |
PAPERCLIP_ALLOWED_ATTACHMENT_TYPES / PAPERCLIP_ATTACHMENT_MAX_BYTES |
— | Upload MIME allow-list & size cap |
RUN_LOG_BASE_PATH / WORKSPACE_OPERATION_LOG_BASE_PATH |
— | Base paths for run / workspace-operation logs |
PAPERCLIP_WORKTREES_DIR / PAPERCLIP_WORKTREE_START_POINT / PAPERCLIP_WORKTREE_NAME |
— | Developer worktree configuration |
PAPERCLIP_TELEMETRY_BACKEND_URL / _TOKEN, PAPERCLIP_FEEDBACK_EXPORT_BACKEND_URL / _TOKEN |
— | Telemetry / feedback export backends |
PAPERCLIP_ENABLE_COMPANY_DELETION |
— | Allow destructive company deletion via API |
6. Authentication & Authorization
6.1 Actor model
The auth middleware resolves a request to one of three actor kinds — board (human user), agent, or anonymous — from whichever credential is present:
| Credential | Actor | Notes |
|---|---|---|
| better-auth session | board | Cookie-based browser session |
| Board API key | board | Issued per board, manageable via /board-api-keys |
| Agent API key | agent | Per-agent keys (/agents/:id/keys) |
| Agent JWT | agent | HS256, configurable TTL/issuer/audience |
| Cloud-tenant token | board (tenant) | PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN |
| Trusted-loopback implicit | board | Only in local_trusted mode on loopback |
6.2 Bootstrapping the first admin
On an authenticated instance, the first administrator (the "CEO") is created from a one-time invite:
paperclipai auth bootstrap-ceo --expires-hours 48
This prints an invite link (claimed via the /bootstrap/claim and /invites/:token flows). --force issues a new invite even if an admin already exists. The CLI reads DATABASE_URL, PAPERCLIP_PUBLIC_URL/PAPERCLIP_AUTH_PUBLIC_BASE_URL, and the better-auth URLs to build the link.
Note
In
local_trustedmode this is unnecessary — the CLI notes that the bootstrap invite is only required forauthenticatedmode (loopback users are implicitly the board).
6.3 Invites, board-claim & CLI auth
- Invites — companies issue invite tokens (
/invites/:token, with onboarding text, logo, and skills index served per-invite); admins revoke them. - Board-claim — a token-based flow (
/board-claim/:token) to claim board access. - CLI auth — challenge/response (
/cli-auth/challenges/:id,/cli-auth/me,/cli-auth/revoke-current) lets the CLI obtain a session without a browser-stored cookie. - Join requests & members — companies expose membership, join-requests, and a user directory.
6.4 Agent credentials
Agents authenticate with per-agent API keys or short-lived JWTs. Keys are created and revoked via /agents/:id/keys. JWTs are signed with PAPERCLIP_AGENT_JWT_SECRET (HS256) and carry the configured issuer/audience and TTL. The CLI's heartbeat run passes an agent bearer token via --api-key.
Warning
A known issue (#7450) exists where
run-stampedwrites can 500 when a gateway JWT'srun_idhas been pruned fromheartbeat_runs(the FK lackedON DELETE SET NULL). If you operate at high run volume, track this in your upgrade notes.
7. Agent Execution Model
7.1 Heartbeat runs
Agents execute in discrete heartbeat runs. A scheduler (HEARTBEAT_SCHEDULER_ENABLED, tick HEARTBEAT_SCHEDULER_INTERVAL_MS, default 30 s) wakes agents that have work; runs are also triggerable manually (POST /agents/:id/wakeup, or POST /agents/:id/heartbeat/invoke) and via the CLI heartbeat run. A run carries a trigger of manual, ping, callback, or system.
Run-related surfaces:
/companies/:companyId/heartbeat-runs,/companies/:companyId/live-runs,/instance/scheduler-heartbeats— fleet-level views./heartbeat-runs/:runIdplus/events,/log,/workspace-operations,/cancel,/watchdog-decisions— per-run inspection and control./issues/:issueId/live-runs,/issues/:issueId/active-run— runs scoped to an issue.
7.2 Watchdog & recovery
Runs are supervised by a watchdog (decisions recorded per run) and a recovery subsystem (server/src/services/recovery/) that handles stalled/dead runs. Issues carry recovery actions (/issues/:id/recovery-actions, /recovery-actions/resolve) and a monitor (/issues/:id/monitor/check-now, /scheduled-retry/retry-now).
Note
Execution locks tie a run to an issue checkout. A frequently requested enhancement (#7458) is a server-side TTL that auto-clears execution locks left behind by dead runs without operator intervention. Until then, an admin can
POST /issues/:id/admin/force-releasea stuck issue.
7.3 Cross-instance / cloud upstreams
/cloud-upstreams/* provides a flow to connect an instance to an upstream and push runs to it (preview, push, cancel, activate). This underpins syncing work between a local instance and a hosted/cloud Paperclip.
8. Work Model: Issues, Approvals, Goals
8.1 Issues
Issues are the core work unit (server/src/routes/issues.ts — the single largest route module). Beyond basic CRUD and search, the issue surface includes:
- Comments & interactions — threaded discussion (
/issues/:id/comments,/interactions), feedback votes and traces. - Documents & work products — versioned documents with lock/unlock, revisions, and inline annotation threads (
/issues/:id/documents/:key/...), plus work products (the deliverables). - Attachments — uploaded files/images, gated by
PAPERCLIP_ALLOWED_ATTACHMENT_TYPES/PAPERCLIP_ATTACHMENT_MAX_BYTES. - Hierarchy & planning — children (
/issues/:id/children) and accepted plan decompositions. - Lifecycle — checkout/release, admin force-release, read/inbox-archive state, labels, and per-issue approvals.
- Tree control —
/issues/:id/tree-control/*and/tree-holds/*coordinate work across an issue tree (holds prevent concurrent conflicting work on related issues).
Important
Several historical 500s on
GET /issues/:id/comments?after=<cursor>(community issues #2612, #3661, #3830) stemmed from aDatevalue being passed where the Postgres driver expected a string in the comment-pagination path. Ensure you run a build where this is fixed; the bug surfaces specifically on theafter=cursor.
8.2 Approvals
Approvals (/companies/:companyId/approvals, /approvals/:id/...) gate sensitive work: approve, reject, resubmit, and a comment thread per approval. They bind to issues (/issues/:id/approvals).
8.3 Goals & dashboard
Goals (/companies/:companyId/goals) track company objectives; the dashboard route aggregates company state for the UI overview.
9. Adapter Subsystem
Adapters are the pluggable agent backends. The registry resolves an adapter type to its implementation; agents reference a type and a model.
| Adapter | Backing SDK / mechanism |
|---|---|
claude-local |
Local Claude agent (ships a quota probe CLI) |
codex-local |
Local Codex agent |
cursor-local |
Local Cursor agent |
cursor-cloud |
@cursor/sdk cloud API |
gemini-local |
Local Gemini agent |
grok-local |
Local Grok agent |
opencode-local |
Local OpenCode agent |
pi-local |
Local Pi agent |
acpx-local |
Agent Client Protocol (acpx, @agentclientprotocol/claude-agent-acp, @zed-industries/codex-acp) |
openclaw-gateway |
WebSocket (ws) gateway to remote agents |
Adapter management API (server/src/routes/adapters.ts): GET /adapters, POST /adapters/install, GET/PATCH/DELETE /adapters/:type, PATCH /adapters/:type/override, POST /adapters/:type/reload, POST /adapters/:type/reinstall, GET /adapters/:type/config-schema, GET /adapters/:type/ui-parser.js. Model discovery is per-company-per-adapter (/companies/:companyId/adapters/:type/models, /model-profiles, /detect-model). The /llms/agent-configuration*.txt endpoints expose adapter configuration to LLM clients.
Tip
Document and operate adapters as interchangeable backends, not as bespoke integrations. A given agent's behavior is defined by its instructions + skills + model; the adapter is just which engine runs it. This is what lets you A/B different engines on the same work.
10. Environments, Workspaces & Sandbox Providers
10.1 The execution surface
- Projects (
/companies/:companyId/projects) own workspaces (/projects/:id/workspaces). - Execution workspaces (
/companies/:companyId/execution-workspaces,/execution-workspaces/:id) are where commands and runtime services actually run; they expose runtime-service/command actions, a close-readiness check, and a workspace-operations log. - Environments (
/companies/:companyId/environments) describe sandbox capabilities, hand out leases (/environments/:id/leases,/environment-leases/:leaseId), and can be probed for health.
10.2 Sandbox providers (plugins)
Sandbox providers are plugins implementing a common provider interface:
| Provider | Package | Backing SDK |
|---|---|---|
| E2B | @paperclipai/plugin-e2b |
e2b |
| Daytona | @paperclipai/plugin-daytona |
@daytonaio/sdk |
| Modal | @paperclipai/plugin-modal |
Modal (token id + secret) |
| Cloudflare | @paperclipai/plugin-cloudflare-sandbox |
@cloudflare/sandbox + a deployed Worker bridge (bridge-template, wrangler) |
| exe-dev | sandbox-providers/exe-dev | exe.dev API (EXE_API_KEY) |
| fake-sandbox | @paperclipai/plugin-fake-sandbox |
Deterministic in-process fake for tests |
10.3 The callback bridge
packages/adapter-utils/src/sandbox-callback-bridge.ts plus the PAPERCLIP_BRIDGE_* settings implement a queue-based bridge so code running inside a sandbox can call back into the Paperclip host (runtime API). The Cloudflare provider additionally ships a Worker bridge-template that proxies between the Cloudflare sandbox and the host. The bridge is the security boundary between untrusted sandboxed execution and the trusted host — protect it with PAPERCLIP_BRIDGE_TOKEN.
Relevant error classes: EnvironmentRunError, PluginSandboxError, CloudflareBridgeError, ExeDevApiError, plus provider SDK errors (SandboxNotFoundError, TimeoutError, CommandExitError).
11. Plugin Runtime & SDK
11.1 Architecture
Plugins extend the platform with UI, tools, jobs, and webhooks. Each plugin declares a manifest, a worker, and (optionally) a UI bundle:
"paperclipPlugin": {
"manifest": "./dist/manifest.js",
"worker": "./dist/worker.js",
"ui": "./dist/ui/"
}
The host runs the worker as a sandboxed JSON-RPC process (packages/plugins/sdk/src/protocol.ts), exchanging messages over the plugin bridge. A capability model (host-client-factory.ts) gates what a worker may do: requests beyond a plugin's grant raise CapabilityDeniedError or InvocationScopeDeniedError; malformed RPC raises JsonRpcParseError / JsonRpcCallError.
11.2 What plugins can contribute
| Surface | API |
|---|---|
| Tools (callable by agents) | /plugins/tools, /plugins/tools/execute |
| UI contributions | /plugins/ui-contributions, static UI at /_plugins/:pluginId/ui/* |
| Background jobs | /plugins/:pluginId/jobs, /jobs/:jobId/runs, /jobs/:jobId/trigger |
| Webhooks | /plugins/:pluginId/webhooks/:endpointKey |
| Bridge data/actions | /plugins/:pluginId/bridge/data|action, /bridge/stream/:channel |
| Config | /plugins/:pluginId/config, /config/test |
| Local-folder dev plugins | /plugins/:pluginId/companies/:companyId/local-folders/* |
| Lifecycle | install, enable/disable, upgrade, health, logs, delete |
11.3 Authoring a plugin
Scaffold with the generator (@paperclipai/create-paperclip-plugin):
npm create paperclip-plugin@latest
A plugin dev server (paperclip-plugin-dev-server) serves the UI during development. First-party reference plugins demonstrate the surface — plugin-kitchen-sink-example intentionally exercises the full API, plugin-hello-world-example is the minimal widget, plugin-orchestration-smoke-example covers orchestration-grade APIs, and plugin-workspace-diff / plugin-llm-wiki / plugin-file-browser-example are production-style features.
Note
Plugins can own database schema and migrations within their own scope (e.g. plugin-llm-wiki ships migrations), keeping plugin tables isolated from core tables.
12. Skills Catalog
packages/skills-catalog/ is a build-time catalog of agent skills, split into bundled (always available — e.g. paperclip-operations/issue-triage, task-planning, software-development/github-pr-workflow, quality/qa-acceptance, docs/doc-maintenance, product/wireframe) and optional (e.g. browser/agent-browser, content/release-announcement, product/design-critique) sets. The catalog builder validates each skill package (it raises ${packageJsonPath} must declare a package version. for malformed entries) and generates an index. Skills are served to agents and clients via /skills/available, /skills/index, /skills/:skillName, and per-company skill endpoints; repo-level skills under skills/ (e.g. paperclip-create-agent, paperclip-create-plugin, terminal-bench-loop) document agent/plugin authoring workflows.
13. Routines & Triggers
Routines (server/src/routes/routines.ts) are scheduled/triggered company workflows with full revision history:
| Surface | Endpoint(s) |
|---|---|
| CRUD | /companies/:companyId/routines, /routines/:id |
| Revisions | /routines/:id/revisions, /revisions/:revisionId/restore |
| Triggers | /routines/:id/triggers, PATCH/DELETE /routine-triggers/:id |
| Execution | /routines/:id/run, /routines/:id/runs |
| Public firing | /routine-triggers/public/:publicId/fire |
The CLI exposes routines disable-all --company-id <id> as an operational kill switch. A frequently requested feature (#7448) is letting manager agents create and manage routines for their subordinates.
14. Costs, Budgets & Finance
Paperclip records cost and finance events and aggregates them many ways (server/src/routes/costs.ts):
| Concern | Endpoints |
|---|---|
| Ingest | POST /companies/:companyId/cost-events, /finance-events |
| Cost summaries | /costs/summary, /costs/by-agent, /by-agent-model, /by-provider, /by-biller, /by-project, /issues/:id/cost-summary |
| Finance ledger | /costs/finance-summary, /finance-by-biller, /finance-by-kind, /finance-events |
| Windows | /costs/window-spend, /costs/quota-windows |
| Budgets | GET /companies/:companyId/budgets/overview, PATCH /companies/:companyId/budgets, PATCH /agents/:agentId/budgets |
Note
Budgets are spend-based today; token-based agent budgets are an open enhancement (#1756). Quota windows align cost accounting with provider rate/usage windows — useful when a provider meters by rolling window rather than absolute spend.
15. Secrets Management
Secrets are company-scoped and provider-backed (server/src/routes/secrets.ts, server/src/secrets/):
- Providers —
local(encrypted file viaPAPERCLIP_SECRETS_MASTER_KEY[_FILE]) or AWS KMS-backed (PAPERCLIP_SECRETS_AWS_*plus standard AWS credential resolution). List/configure providers, mark one default, and health-check them (/secret-providers,/secret-provider-configs). - Secrets — create, update, rotate, delete (
/companies/:companyId/secrets,/secrets/:id,/secrets/:id/rotate), with audit surfaces/secrets/:id/usageand/secrets/:id/access-events. - Strict mode —
PAPERCLIP_SECRETS_STRICT_MODEmakes any decryption failure fatal, which surfaces key-rotation problems immediately instead of silently degrading.
Caution
The local provider's security reduces to the secrecy of the master key. Protect
PAPERCLIP_SECRETS_MASTER_KEY_FILEwith strict file permissions and disk encryption, or use the AWS KMS provider so the data-encryption keys never live in plaintext on the host.
16. MCP Server
packages/mcp-server/ publishes the paperclip-mcp-server binary (built on @modelcontextprotocol/sdk + Zod). It lets an external MCP-aware LLM client read and act on a Paperclip instance over stdio. Point your client's command at it and provide the instance base URL and an API key. Server-side, /llms/*.txt endpoints expose agent/adapter configuration in an LLM-friendly form.
17. Data Model & Database
The data layer is Drizzle ORM over PostgreSQL, with 86 schema files in packages/db/src/schema/ and 94 migrations in packages/db/src/migrations/.
17.1 Major entity groups
| Group | Represents |
|---|---|
| Companies & access | companies, members, invites, board API keys, branding |
| Agents | agents, agent keys, configurations + revisions, runtime state, task sessions, skills, hires |
| Heartbeat runs | run records, events, logs, watchdog decisions, workspace operations |
| Issues | issues, comments, interactions, documents (+revisions/annotations), work products, attachments, approvals, labels, tree control/holds, read & inbox state |
| Projects & environments | projects, workspaces, execution workspaces, environments, leases, probes |
| Routines | routines, triggers, revisions, runs |
| Costs & finance | cost events, finance events, budgets, quota windows |
| Secrets | secrets, secret provider configs, access events |
| Plugins | plugin registrations, configs, jobs/runs, webhooks, data |
| Goals, approvals, activity | goals, approvals (+comments), activity feed |
| Auth | better-auth users/sessions/accounts |
| Instance | instance settings, database backups |
17.2 Migrations & embedded Postgres
Migrations are generated with drizzle-kit (pnpm --filter @paperclipai/db generate) and applied via pnpm --filter @paperclipai/db migrate or automatically on startup (PAPERCLIP_MIGRATION_AUTO_APPLY). A check:migrations guard enforces migration numbering. Embedded Postgres is a patched embedded-postgres@18.1.0-beta.16 (see patches/); its port is set by PAPERCLIP_EMBEDDED_POSTGRES_PORT.
17.3 Backup & restore
packages/db/src/backup-lib.ts drives backups using pg_dump/psql (paths overridable via PAPERCLIP_PG_DUMP_PATH / PAPERCLIP_PSQL_PATH). The CLI db:backup supports --dir, --retention-days, --filename-prefix, and --json; the API exposes POST /instance/database-backups. Schedule db:backup via cron for unattended backups.
18. API Reference
The server mounts ~328 routes across 40 route modules under the /api prefix, with an auto-generated OpenAPI document at GET /api/openapi.json (built from an OpenAPIRegistry). Treat that document as the authoritative, always-current contract; the table below is the resource map.
| Resource module | Representative paths |
|---|---|
health |
GET /health, POST /dev-server/restart |
auth |
GET /get-session, GET/PATCH /profile |
access |
invites, board-claim, bootstrap claim, CLI-auth, board API keys, skills, members, admin users |
agents |
agent CRUD, hire (POST /companies/:companyId/agent-hires), pause/resume/approve/terminate, permissions, instructions bundle, config revisions/rollback, keys, wakeup, heartbeat invoke, org chart (/companies/:companyId/org[.svg/.png]) |
activity |
company & issue activity, issue runs, heartbeat-run issues |
adapters |
list/install/patch/reload/reinstall, config-schema, models/model-profiles/detect-model |
issues |
CRUD, search, comments, interactions, documents, work products, attachments, approvals, children, plan decompositions, checkout/release, admin force-release, tree-control/holds, recovery, monitor |
approvals |
CRUD, approve/reject/resubmit, comments |
companies |
CRUD, stats, export/import, branding, archive |
company-skills / instance-settings |
skill catalog, general/experimental instance settings |
projects / execution-workspaces / environments |
projects+workspaces, execution workspaces, environments+leases+probes |
routines |
CRUD, triggers, revisions, run, public fire |
costs |
cost/finance events, summaries, budgets, quota windows |
secrets |
providers, configs, secrets, rotate, usage, access events |
plugins |
install/enable/disable/upgrade, config, health, logs, tools, jobs, webhooks, bridge, UI |
goals / dashboard / assets |
goals, dashboard data, image/logo assets |
cloud-upstreams |
connect, push-runs lifecycle |
instance-database-backups |
trigger DB backups |
llms |
agent-configuration/icons text for LLM clients |
openapi |
GET /openapi.json |
Tip
Generate clients from
/api/openapi.jsonrather than hand-coding paths — the route surface is large and evolving, and the OpenAPI document is regenerated from the registry on every build. Many resources are mounted under a company scope (/companies/:companyId/...) while individual records are addressed by their own id (/issues/:id,/agents/:id,/routines/:id).
19. Security
19.1 Posture checklist
- [ ]
BETTER_AUTH_SECRETset (256-bit random) for any non-loopback deployment. - [ ] A distinct
PAPERCLIP_AGENT_JWT_SECRETfrom the auth secret. - [ ]
PAPERCLIP_DEPLOYMENT_MODE=authenticatedwhenever exposure ≠ loopback. - [ ]
PAPERCLIP_ALLOWED_HOSTNAMES+BETTER_AUTH_TRUSTED_ORIGINSconstrained to known hosts/origins. - [ ]
PAPERCLIP_AUTH_DISABLE_SIGN_UP=trueonce your initial users exist. - [ ]
PAPERCLIP_BRIDGE_TOKENset so only authorized processes can use the sandbox-callback bridge. - [ ] Secrets via AWS KMS (
PAPERCLIP_SECRETS_PROVIDER=AWS) or a tightly-permissioned master key file; considerPAPERCLIP_SECRETS_STRICT_MODE. - [ ] Attachment MIME allow-list and size cap configured.
- [ ] Untrusted agent code executed only in cloud sandboxes / the untrusted-review container, never directly on the host.
19.2 Built-in protections
- Log redaction (
server/src/log-redaction.ts) scrubs identifiers (USER,LOGNAME,USERNAME, etc.) from logs. - Hardened review container (
docker/untrusted-review/) drops all capabilities and forbids privilege escalation. - Capability-gated plugins — workers cannot exceed their declared grants (
CapabilityDeniedError). - Hostname allow-listing rejects requests from unexpected hosts; invalid hostnames raise
Invalid hostname: <raw>. - Loopback enforcement —
local_trustedmode is rejected on any non-loopback host. - HTTP error mapping — domain errors extend
HttpError(server/src/errors.ts) for consistent status codes.
19.3 Handling a malicious or compromised agent
Agents are privileged actors. Pause or terminate a misbehaving agent immediately (/agents/:id/pause, /terminate), revoke its keys (DELETE /agents/:id/keys/:keyId), force-release any issues it holds, and review its /secrets/:id/access-events to see what it touched. Run untrusted execution only in isolated sandboxes so a compromised run cannot reach the host.
20. Operations Runbook
| Task | Command / endpoint |
|---|---|
| Start an instance | paperclipai run -i <instance> |
| Diagnose / repair | paperclipai doctor --repair |
| Inspect resolved env | paperclipai env |
| Back up the DB | paperclipai db:backup --dir <path> --retention-days <n> --json |
| Allow a hostname | paperclipai allowed-hostname add <host> |
| Health check | GET /api/health |
| Pause all scheduled work | paperclipai routines disable-all --company-id <id> |
| Force-release a stuck issue | POST /api/issues/:id/admin/force-release |
| Cancel a run | POST /api/heartbeat-runs/:runId/cancel |
20.1 Developer worktrees
The worktree command family runs isolated copies of an instance side-by-side — each with its own repo-local config, isolated data dir, server port, and embedded-Postgres port — so developers can work on multiple branches without colliding state:
| Subcommand | Purpose |
|---|---|
worktree:make |
Create a new branch + isolated instance (seed from a source instance; --seed-mode minimal|full) |
init |
Initialize an isolated instance in the current repo |
worktree:list |
List worktree instances |
env |
Print shell exports (or --json) for a worktree |
worktree:merge-history |
Import issues/comments from another worktree (--scope, --dry/--apply) |
reseed |
Re-seed a target worktree's DB from a source (guards against live targets) |
repair |
Repair/bootstrap a worktree's metadata/config |
worktree:cleanup |
Remove a worktree (with safety checks unless --force) |
Warning
reseedandworktree:cleanupare destructive. They guard against operating on a live target DB;--allow-live-target/--forceoverride those guards — use them deliberately.--preserve-live-workkeeps copied agent timers and assigned open issues instead of quarantining them.
20.2 env-lab
env-lab up|status|down|doctor manages local fixture environments for testing adapter/sandbox configurations without touching production data.
20.3 Performance tuning
- Database is the first bottleneck. Each heartbeat cycle queries active agents; under many concurrent agents, watch PostgreSQL connection count (
SELECT count(*) FROM pg_stat_activity;). Move to an external Postgres with adequatemax_connectionsbefore scaling agent count. - Scheduler cadence. Lower
HEARTBEAT_SCHEDULER_INTERVAL_MSfor snappier response, raise it to reduce database churn. - Storage I/O. Use S3 storage to remove local disk pressure for attachment-heavy workloads.
- Embedded-Postgres diagnostics.
PAPERCLIP_EMBEDDED_POSTGRES_VERBOSE=truelogs database activity for slow-query hunting.
21. Build, Test & Release
| Activity | Command |
|---|---|
| Install | pnpm install |
| Build all | pnpm build (runs workspace-link preflight, then pnpm -r build) |
| Typecheck | pnpm typecheck |
| Unit/integration tests | pnpm test (stable Vitest runner) |
| E2E | pnpm test:e2e (Playwright) |
| Release smoke | pnpm test:release-smoke |
| Dev server | pnpm dev (server with tsx watch) / pnpm dev:ui |
| DB generate/migrate | pnpm db:generate / pnpm db:migrate |
| Eval (promptfoo) | pnpm evals:smoke |
Releases are scripted (scripts/release.sh with canary/stable channels, GitHub release + rollback helpers) and the npm CLI is bundled via esbuild (scripts/build-npm.sh). The repo enforces guardrails such as check:no-git-push and forbidden-token checks. Test/CI surfaces include Storybook for UI components and a release-smoke Playwright suite that exercises Docker auth/onboarding.
Error class hierarchy (selected)
Error
├── HttpError server/src/errors.ts (HTTP status mapping)
├── EnvironmentRunError services/environment-run-orchestrator.ts
├── PluginSandboxError services/plugin-runtime-sandbox.ts
├── CapabilityDeniedError plugins/sdk/host-client-factory.ts
├── InvocationScopeDeniedError plugins/sdk/host-client-factory.ts
├── JsonRpcParseError / plugins/sdk/protocol.ts
│ JsonRpcCallError
├── CloudflareBridgeError sandbox-providers/cloudflare
├── ExeDevApiError sandbox-providers/exe-dev
├── PaperclipApiError mcp-server/client.ts
└── ApiError / AuthApiError ui/src/api
22. Code Quality & Conventions
- Strict TypeScript, ESM throughout. Every package is
"type": "module", targets TS 5.7, and publishes typed entry points; the CLI bundles workspace deps via esbuild. - Zod as the validation spine.
packages/shared/src/validators/(36 files) holds input schemas reused across server, CLI, and UI, and feeds the OpenAPI registry — request validation and API docs share one source of truth. - Service-oriented server. 117 service files concentrate business logic behind thin route handlers; the
recovery/subdirectory isolates run-recovery concerns. - Adapter/provider symmetry. Both agent adapters and sandbox providers follow a registry + uniform-interface pattern, which is what makes them swappable.
- Test breadth. Vitest unit/integration (
server/src/__tests__alone has 261 files), Playwright E2E + release-smoke, promptfoo evals, and Storybook stories.
Note
The largest modules —
server/src/services(~2.7M chars),server/src/routes(~1.06M chars),ui/src/lib(~0.68M chars),packages/adapter-utils/src(~0.44M chars), andpackages/plugins/sdk/src(~0.36M chars) — are the natural focus points for anyone learning the codebase or planning refactors.
23. Known Limitations & Roadmap Signals
Drawn from the issue tracker at this commit:
| Area | Signal |
|---|---|
| Execution locks | Server-side TTL to auto-clear dead-run locks without an operator (#7458) |
| Agent FK integrity | run-stamped 500 when a gateway JWT's run_id is pruned from heartbeat_runs (no ON DELETE SET NULL) (#7450) |
| Budgets | Token-based agent budgets as an alternative to spend-based (#1756) |
| Routines | Let manager agents create/manage routines for subordinates (#7448) |
| Providers | OpenRouter support (#1068) |
| Robustness | Agents can loop indefinitely with some Ollama models (#4678) — cap with budgets + heartbeat timeouts |
| Comments API | Date-as-bind-param 500 on /issues/:id/comments?after= (#2612/#3661/#3830) — fixed in current builds; verify on upgrade |
Scaling beyond a single node is the largest architectural limitation: shared embedded Postgres and the file-backed bridge queue assume one owning process. Production scale means external Postgres + S3 storage + vertical sizing, and (today) one active server.
24. Quick Reference
| Item | Value |
|---|---|
| Default port | 3100 (PORT / PAPERCLIP_LISTEN_PORT) |
| In-process bind | 127.0.0.1 (presets: loopback/lan/tailnet) |
| Health | GET /api/health |
| OpenAPI | GET /api/openapi.json |
| Data root | PAPERCLIP_HOME (default ~/.paperclip) |
| Auth modes | local_trusted / authenticated × private / public (default local_trusted + private) |
| DB | PostgreSQL via Drizzle (embedded or DATABASE_URL) |
| Scheduler tick | HEARTBEAT_SCHEDULER_INTERVAL_MS (default 30000) |
| First admin | paperclipai auth bootstrap-ceo |
| Kill switch | paperclipai routines disable-all --company-id <id> |
| Force-release issue | POST /api/issues/:id/admin/force-release |
For user-facing workflows, the adapter/sandbox catalogs, and the CLI day-to-day, see the Paperclip User Guide.