CrewAI Admin & Architecture Guide

This guide is for engineers, architects, and platform operators who need to understand how CrewAI is built, configure it exhaustively, deploy it, secure it, and extend it. Where the User Guide teaches you to use the framework, this guide explains how it works internally — the monorepo layout, the execution engine, the event/observability system, the LLM provider abstraction, the memory/knowledge/RAG subsystems, security, and the full configuration surface.

Note

CrewAI is a framework, not a deployable application. "Deployment" here means publishing a crew you built to the CrewAI AMP (Agent Management Platform) cloud, or running it inside your own infrastructure. The framework itself has no server you stand up — it runs wherever your Python runs.

1. Executive Summary

CrewAI is a Python multi-agent orchestration framework distributed as the PyPI package crewai (version 1.14.7a1 at the documented commit). It is a UV workspace monorepo of six packages, with crewai (the core library) as the primary product and crewai-cli, crewai-core, crewai-tools, crewai-files, and crewai-devtools as companions.

Design philosophy: a lean, standalone framework independent of LangChain, exposing both high-level simplicity (declarative agents/tasks/crews) and low-level control (Flows, custom executors, native prompts).

2. Architecture

2.1 Monorepo layout

CrewAI is a uv workspace (tool.uv.workspace in the root pyproject.toml). Each member is independently versioned and published, but they share one lockfile and dependency-override policy.

crewAI/  (workspace root: ruff, mypy, bandit, pytest config, uv overrides)
├── lib/crewai/         crewai            ← THE core library (Agent/Task/Crew/Flow/LLM)
├── lib/cli/            crewai-cli        ← the `crewai` command (Click)
├── lib/crewai-core/    crewai-core       ← shared auth, token manager, printer
├── lib/crewai-tools/   crewai-tools      ← ~80 prebuilt tools (optional [tools] extra)
├── lib/crewai-files/   crewai-files      ← native file-input handling
├── lib/devtools/       crewai-devtools   ← developer tooling
├── docs/               MkDocs site (en/ar/ko/pt-BR — multilingual)
└── scripts/            maintenance scripts

Dependency direction: crewai depends on crewai-core and crewai-cli; crewai-cli depends on crewai-core; crewai-tools and crewai-files are optional and depend on crewai. The crewai console-script entry point is crewai = "crewai_cli.cli:crewai".

2.2 The core package (lib/crewai/src/crewai/)

crewai/
├── __init__.py        Public API + Pydantic forward-ref rebuild orchestration
├── crew.py            Crew (2,351 LOC) — orchestration, kickoff, processes
├── task.py            Task (1,463 LOC) — execution, guardrails, output export
├── agent/             Agent (core.py, 1,952 LOC) + planning_config
├── agents/            BaseAgent, executors (crew_agent_executor), cache, adapters
├── lite_agent.py      Lightweight standalone agent
├── llm.py             LLM (2,525 LOC) — provider routing + LiteLLM
├── llms/              base_llm + providers/{openai,anthropic,azure,bedrock,gemini,...}
├── flow/              Flow DSL (dsl.py), runtime engine (runtime.py), persistence
├── process.py         Process enum (sequential | hierarchical)
├── tools/             BaseTool, structured_tool, agent_tools, memory_tools
├── memory/            unified_memory, memory_scope, storage
├── knowledge/         Knowledge + source/{string,text,pdf,csv,json,excel,docling}
├── rag/               Embeddings + vector stores (chromadb, qdrant, lancedb)
├── events/            Event bus, event types, listeners (incl. tracing)
├── security/          Fingerprint, SecurityConfig
├── a2a/               Agent-to-Agent protocol (remote delegation)
├── mcp/               Model Context Protocol transports + tool resolver
├── skills/            Agent skills loader/registry
├── state/             Runtime state, checkpoint config
├── telemetry/         Anonymous OpenTelemetry telemetry
├── experimental/      AgentExecutor (new default), evaluation
├── project/           @CrewBase / @agent / @task / @crew decorators
└── utilities/         LLM factory, converters, i18n, RPM controller, prompts

2.3 Execution model — Crews

A Crew is a Pydantic model that holds agents, tasks, and a process. kickoff(inputs) does the following:

crew.kickoff(inputs)
  ├─ apply_checkpoint()                  resume if from_checkpoint set
  ├─ enable_agent_streaming()            if stream=True
  ├─ attach OpenTelemetry baggage        crew_context (id, key)
  ├─ prepare_kickoff()                   interpolate inputs into tasks/agents
  ├─ process == sequential →  _run_sequential_process()
  │     for each task: task.execute_sync(agent) → TaskOutput
  ├─ process == hierarchical → _run_hierarchical_process()
  │     manager agent (manager_llm/manager_agent) delegates + validates
  ├─ after_kickoff_callbacks             post-process the CrewOutput
  ├─ calculate_usage_metrics()           token accounting
  └─ drain memory writes, clear files, detach baggage

Each Task.execute_sync() calls Agent.execute_task(), which: builds the prompt (description + expected_output + memory + knowledge + tools), runs the AgentExecutor loop (reason → call tool → observe → repeat up to max_iter), applies guardrails with retry, exports structured output, and emits events at every stage.

Note

The agent execution loop moved from CrewAgentExecutor to crewai.experimental.AgentExecutor, which is now the default (Agent.executor_class). CrewAgentExecutor is deprecated but still selectable for backward compatibility.

2.4 Execution model — Flows

Flows are defined in crewai.flow, split by concern:

A Flow[StateModel] carries a typed state object. The engine builds a dependency graph from the decorators and fires methods as their triggers are satisfied. StateProxy/LockedDictProxy/LockedListProxy wrap mutable state so concurrent writes are controlled. crewai.flow.flow re-exports the public surface for backward compatibility.

2.5 The event bus and observability

A singleton crewai_event_bus is the backbone of CrewAI's observability. Every meaningful action emits a typed event:

Listeners subscribe to the bus. Built-in listeners include the console EventListener (verbose output) and TraceCollectionListener (sends traces to CrewAI AMP when enabled). The event record also backs checkpointing: runtime state is reconstructed from the recorded event graph on resume.

3. System Requirements

Important

The root pyproject.toml carries an extensive tool.uv.override-dependencies block that force-upgrades transitive dependencies past known CVEs (e.g. urllib3>=2.7.0, cryptography>=46.0.7, gitpython>=3.1.50, langchain-core>=1.3.3, starlette>=1.0.1, pypdf>=6.10.2). When vendoring or re-locking, preserve these overrides — they are security-critical. See §7.4.

4. Configuration Reference

This is the exhaustive parameter reference. The User Guide covers the everyday subset; everything is here.

4.1 Agent — full parameter list

Agent extends BaseAgent. Fields below combine both.

Warning

The built-in code interpreter was removed. allow_code_execution, code_execution_mode, and get_code_execution_tools() are no-ops that emit DeprecationWarning. For code execution, use a sandbox tool: E2BPythonTool or DaytonaPythonTool from crewai-tools.

4.2 Task — full parameter list

Caution

output_file is security-validated at construction: paths containing .., leading ~/$, or shell metacharacters (| > < & ;) raise ValueError. Template variables ({var}) are permitted only if each is a valid identifier. This prevents path-traversal and shell-injection via task config.

4.3 Crew — full parameter list

Crew validators enforce: a manager is present for hierarchical; the manager is not in agents; at least one of config/tasks/agents is set; id cannot be user-set.

4.4 LLM — parameters

LLM (llm_type="litellm") extends BaseLLM. Construction uses a __new__ factory that routes to a native provider class or the LiteLLM path.

4.5 YAML configuration

When you use @CrewBase, config/agents.yaml and config/tasks.yaml are loaded into self.agents_config / self.tasks_config. Keys map directly to Agent/Task parameters. Placeholders like {topic} are interpolated from the inputs dict at kickoff. A scaffolded project's pyproject.toml carries a [tool.crewai] table with type = "crew" (or "flow").

5. LLM Provider Architecture

LLM.__new__ is a routing factory:

LLM(model=..., provider=?, ...)
  1. explicit provider kwarg          → native provider class
  2. "provider/model" prefix          → map prefix → canonical provider
        openai|anthropic|claude|azure|google|gemini|bedrock|aws → native
        openrouter|deepseek|ollama|cerebras|hosted_vllm|dashscope → LiteLLM
  3. bare model name                  → infer provider from constants
  → if native + supported + not is_litellm → instantiate native provider class
  → else → fall back to LiteLLM (requires crewai[litellm])

All providers implement the BaseLLM interface (call, supports_function_calling, supports_multimodal, get_token_usage_summary). create_llm() (utilities/llm_utils.py) resolves a string or dict into the right instance. LLMs serialize to/from JSON via an llm_type registry (base/litellm/openai/anthropic/azure/bedrock/gemini) so crews can be checkpointed and restored.

Note

LLM hooks/interceptors (crewai.hooks.LLMCallHookContext, crewai.llms.hooks.BaseInterceptor) let you intercept and modify LLM calls — useful for governance, redaction, or custom routing. This is the extension point behind several open feature requests (e.g. tool-call authorization middleware, issues #5888/#4877/#6025).

6. Memory, Knowledge, and RAG

6.1 Unified memory

crewai.memory.unified_memory.Memory is the single memory abstraction (backed by LanceDB). Crews namespace memory under /crew/<name>. Scoped views (MemoryScope, MemorySlice) let agents share or partition a backing store. Memory is enabled with Crew(memory=True) or Agent(memory=True); agents fall back to crew memory when they have none.

Memory operations: recall(query, limit), extract_memories(text), remember_many(items), drain_writes() (flushed at the end of kickoff). The crewai memory TUI browses scopes and recall results; crewai reset-memories clears stores.

6.2 Knowledge

crewai.knowledge.Knowledge ingests BaseKnowledgeSource documents and exposes query() for retrieval-augmented grounding. Built-in sources:

Agents generate a knowledge-search query from the task, then retrieve matches and inject them into the prompt. Knowledge config (KnowledgeConfig) tunes result limits and score thresholds.

6.3 RAG and embeddings

crewai.rag provides the embeddings callable interface and vector-store backends (ChromaDB default, Qdrant, LanceDB). EmbedderConfig selects a provider (openai, google, cohere, ollama, huggingface, voyageai, bedrock, instructor, sentence-transformer, onnx, and more under rag/embeddings/providers/). The RAG config is global per process and validated before use (rag/config/utils.py raises clear errors if unset).

7. Security

7.1 Fingerprinting

Every Agent, Task, and Crew carries a SecurityConfig with a Fingerprint (crewai.security). Fingerprints provide stable identity/provenance for entities. A fingerprint seed must be non-empty (ValueError otherwise), and the fingerprint type is validated.

7.2 Guardrails

Guardrails (utilities/guardrail.py) validate task/agent output before it propagates. A guardrail is a callable returning (success: bool, result_or_error) or a natural-language string evaluated by an LLM (LLMGuardrail). Failures trigger bounded retries (guardrail_max_retries, default 3) with the error fed back to the agent. Guardrails are the recommended control point for output-policy enforcement.

7.3 Agent-to-Agent (A2A) auth

The crewai.a2a subsystem implements remote agent delegation over the A2A protocol (crewai[a2a] extra). It supports multiple auth schemes — BearerTokenAuth, HTTPBasicAuth, APIKeyAuth (header location), OAuth2 — with explicit, source-enforced constraints (e.g. gRPC transport requires header-based API-key auth; HTTPDigestAuth is rejected on gRPC). Transports include HTTP and gRPC (gRPC needs a2a-sdk[grpc]). Enterprise auth providers under crewai-core cover Auth0, Entra ID, Keycloak, Okta, and WorkOS (each requires audience/client-id/domain configuration).

7.4 Supply-chain hardening

The workspace pins transitive dependencies past known CVEs via tool.uv.override-dependencies. Examples enforced at the documented commit: urllib3>=2.7.0, cryptography>=46.0.7, gitpython>=3.1.50, langchain-core>=1.3.3, langchain-text-splitters>=1.1.2, transformers>=5.4.0, pypdf>=6.10.2, python-multipart>=0.0.27, starlette>=1.0.1, authlib>=1.6.12, paramiko>=5.0.0, uv>=0.11.15, langsmith>=0.8.0. CI runs bandit and pip-audit.

Important

The crewai-files source loaders enforce defense-in-depth on file inputs: path-traversal rejection, symlink-escape detection, URL-scheme validation, and a max-size guard (FileTooLargeError). When accepting user-supplied files, prefer these loaders over raw file reads.

7.5 File-input safety

crewai-files validates every source: rejects path traversal (Path traversal not allowed), symlink escapes (Symlink escapes allowed directory), unsupported URL schemes, and oversized files. Treat plugin/file-upload surfaces as the primary untrusted-input boundary.

8. Telemetry, Tracing, and Observability

8.1 Anonymous telemetry

CrewAI sends anonymous usage telemetry via OpenTelemetry (crewai.telemetry). Collected: CrewAI version, Python version, general OS info, number of agents/tasks, the process used, whether memory/delegation is on, parallel-vs-sequential execution, the LLM provider, agent roles, and tool names. Never collected: prompts, task descriptions, backstories, goals, tool inputs/outputs, API calls, agent data, or secrets — unless the user sets share_crew=True, which opts into detailed sharing (goal/backstory/context/output).

8.2 Tracing (CrewAI AMP)

TraceCollectionListener forwards execution traces to CrewAI AMP when enabled. Tracing is controlled three ways (precedence: env > per-crew > user setting):

8.3 Logging and metrics

Crew(output_log_file=...) writes execution logs. Crew(verbose=True) streams a rich console view via EventListener. CrewOutput.token_usage and crew.calculate_usage_metrics() expose token accounting (UsageMetrics).

9. CLI Reference (Complete)

The crewai command is a Click group (crewai_cli.cli). Full command tree:

9.1 Top-level commands

9.2 Command groups

Note

crewai version reports _get_cli_version(), which falls back from the crewai package version to crewai-cli to "unknown". The console-script entry (crewai = crewai_cli.cli:crewai) is declared in both lib/crewai/pyproject.toml and lib/cli/pyproject.toml.

10. Deployment & Operations

10.1 Deploying to CrewAI AMP

CrewAI AMP is the managed control plane (tracing, observability, deployments). Typical flow:

crewai login                 # authenticate
crewai deploy validate       # run pre-deploy checks locally (no platform call)
crewai deploy create -y      # create a deployment
crewai deploy push           # push the crew
crewai deploy status         # check status
crewai deploy logs           # tail logs

crewai deploy validate runs the same blocking checks that create/push run automatically, exiting non-zero on failure — wire it into CI.

10.2 Triggers

crewai triggers list enumerates available integration triggers; crewai triggers run app_slug/trigger_slug executes the crew with a trigger payload. The payload is injected into the first task's description (or per-task via allow_crewai_trigger_context). Scaffolded projects expose a run_with_trigger entry point that parses a JSON payload from argv into inputs["crewai_trigger_payload"].

10.3 Self-hosting a crew

To run inside your own infra, package the project (uv build), set the required provider API keys as environment variables, and invoke your entry point (crew().kickoff()), a scheduler, or a web handler. There is no CrewAI server to operate — the framework runs in-process. For long-running or resumable jobs, enable checkpointing.

10.4 Checkpointing and resume

With checkpoint=True (or a CheckpointConfig) on a Crew/Agent, execution state is persisted (SQLite or directory). Resume with crew.kickoff(from_checkpoint=...), Crew.from_checkpoint(config), or Crew.fork(config, branch=...). The crewai checkpoint CLI inspects, resumes, diffs, and prunes checkpoints. State is reconstructed from the recorded event graph and re-bound memory views.

10.5 Operations checklist

• [ ] Pin CrewAI and provider extras; preserve tool.uv.override-dependencies.
• [ ] Set provider API keys via environment, never in code.
• [ ] Decide telemetry posture (OTEL_SDK_DISABLED) and tracing (CREWAI_TRACING_ENABLED).
• [ ] Set max_rpm / max_execution_time to bound cost and runtime.
• [ ] Enable checkpointing for long jobs.
• [ ] Use guardrails for output-policy enforcement.
• [ ] Run crewai deploy validate in CI before deploying.

11. Internals & Module Reference

The crewai.__init__ module does substantial work at import time: it lazily imports heavy modules (Memory → LanceDB), suppresses Pydantic deprecation warnings, and orchestrates model_rebuild() across Agent/Crew/Task/Flow/executors to resolve circular forward references — necessary because these models reference each other.

12. Testing Strategy

The workspace uses pytest with strict, reproducible configuration (root pyproject.toml):

LLM interactions are recorded as VCR cassettes so tests are deterministic and offline. Type checking is mypy --strict (uvx mypy src); linting/formatting is ruff with an extensive rule set including bandit (S) security rules. Contributors run pre-commit install, uv run pytest ., and uvx mypy src.

13. Extending CrewAI

14. Potential Improvements & Roadmap Signals

Drawn from open community issues and code observations at this commit:

• Governance / authorization middleware for tool calls is requested (issues #5888, #4877, #6025, #6024). The BaseInterceptor/LLM-hooks system is the natural integration seam; a first-class pre-tool-call authorization hook would close these.
• Behavioral trust scoring between agents (#5789) and memory-poisoning protection (#6021, OWASP Agent Memory Guard) are requested security enhancements relevant to the memory and A2A subsystems.
• Python 3.14 support (#5109) — currently capped at <3.14; the PyTorch-nightly index is already wired for 3.13.
• External knowledge directories (#3519) — broaden BaseFileKnowledgeSource path handling.
• Deprecation cleanup — CrewAgentExecutor, function_calling_llm, reasoning/max_reasoning_attempts, multimodal, allow_code_execution/code_execution_mode, and Task.max_retries are all slated for removal; consolidating on AgentExecutor, planning_config, native file inputs, sandbox tools, and guardrail_max_retries will reduce surface area.

15. Quick Reference

Public API (crewai)

from crewai import (
    Agent, Crew, Task, Process, Flow, LLM, BaseLLM,
    Knowledge, Memory, LLMGuardrail, PlanningConfig,
    CrewOutput, TaskOutput, ExecutionContext,
)
from crewai.flow.flow import Flow, start, listen, router, or_, and_
from crewai.project import CrewBase, agent, task, crew
from crewai.tools import BaseTool

Process selection

Key environment variables

Repository facts