# AWS CLI Administrator & Developer Guide

## 1. Architecture Overview

AWS CLI v1 (version 1.44.85) is a universal command-line interface for interacting with Amazon Web Services. Built in Python, it wraps the botocore SDK with a full-featured CLI layer including argument parsing, output formatting, help generation, and a plugin/event system for extensibility.

> [!NOTE]
> This document covers the **v1 branch** of AWS CLI. AWS CLI v2 is a separate codebase with different architecture decisions (bundled Python, auto-prompt, SSO integration). V1 remains actively maintained.

### High-Level Component Diagram

```
┌─────────────────────────────────────────────────────────────────────┐
│                         User Shell                                   │
│   $ aws <service> <operation> [--params]                            │
└────────────────────────────┬────────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────────┐
│  bin/aws  →  awscli/__main__.py  →  awscli.clidriver.main()        │
└────────────────────────────┬────────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      CLIDriver                                       │
│  ┌──────────────┐  ┌───────────────┐  ┌────────────────────────┐   │
│  │ MainArgParser│  │ Command Table │  │ botocore Session       │   │
│  │              │  │ (from models) │  │  - credentials         │   │
│  └──────┬───────┘  └───────┬───────┘  │  - config             │   │
│         │                   │          │  - event system        │   │
│         ▼                   ▼          │  - service models      │   │
│  ┌──────────────────────────────┐     └────────────┬───────────┘   │
│  │     ServiceCommand           │                  │               │
│  │       └─ ServiceOperation    │◄─────────────────┘               │
│  │            └─ botocore call  │                                   │
│  └──────────────────────────────┘                                   │
└─────────────────────────────────────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────────┐
│                   Output Pipeline                                    │
│  ┌──────────┐  ┌──────────┐  ┌────────────┐  ┌──────────────────┐ │
│  │ JSON     │  │ Text     │  │ Table      │  │ YAML (optional)  │ │
│  │Formatter │  │Formatter │  │Formatter   │  │ Formatter        │ │
│  └──────────┘  └──────────┘  └────────────┘  └──────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```

### Boot Sequence

1. `bin/aws` invokes `awscli/__main__.py`
2. `main()` calls `create_clidriver()`
3. `create_clidriver()` instantiates a botocore `Session`
4. Plugins are loaded from `~/.aws/config` `[plugins]` section
5. `CLIDriver` is constructed with the session
6. `CLIDriver.main()` parses top-level args, locates the service command
7. `ServiceCommand.__call__()` parses the operation, delegates to `ServiceOperation`
8. `ServiceOperation.__call__()` invokes the botocore API call
9. Response passes through the formatter and is written to stdout

### Event-Driven Extensibility

The entire CLI is wired through botocore's event system. Events fire at every stage:

| Event | When Fired |
|-------|-----------|
| `building-command-table.main` | Command table construction (top-level) |
| `building-command-table.<service>` | Service sub-command table construction |
| `building-argument-table.<service>.<operation>` | Operation arguments populated |
| `process-cli-arg.<service>.<operation>` | Individual argument processing |
| `calling-command.<service>.<operation>` | Just before API dispatch |
| `load-cli-arg` | Loading paramfile references (`file://`, `fileb://`) |
| `session-initialized` | Session ready (plugin hook point) |
| `doc-*` | Various documentation generation events |

Customizations in `awscli/customizations/` register handlers for these events to modify behavior, inject arguments, rewrite parameters, or add entirely new commands.

---

## 2. System Requirements

### Runtime Requirements

| Component | Requirement |
|-----------|------------|
| Python | 3.9 or later |
| OS | Linux, macOS, Windows (any platform with Python 3.9+) |
| Disk | ~50 MB installed (including botocore service models) |
| Network | HTTPS outbound to AWS endpoints (port 443) |
| Shell | bash/zsh (Unix) or cmd/PowerShell (Windows) for completions |

### Python Dependencies

| Package | Version Constraint | Purpose |
|---------|-------------------|---------|
| botocore | ==1.42.95 | AWS SDK core: service models, request signing, credential resolution, sessions |
| docutils | >=0.18.1, <=0.19 | reStructuredText processing for help page generation |
| s3transfer | >=0.16.0, <0.17.0 | High-performance multipart S3 file transfers |
| PyYAML | >=3.10, <6.1 | YAML config parsing |
| colorama | >=0.2.5, <0.4.7 | Cross-platform colored terminal output |
| rsa | >=3.1.2, <4.8 | RSA signing for CloudFront presigned URLs |

> [!IMPORTANT]
> The botocore version is pinned exactly (`==1.42.95`). Upgrading botocore independently will break API compatibility. Always upgrade aws-cli and botocore together.

### Build/Development Requirements

| Tool | Purpose |
|------|---------|
| pytest | Test runner |
| isort | Import sorting |
| ruff | Linting and formatting |
| pre-commit | Git hook management |
| groff or mandoc | Man page rendering (Unix help display) |

---

## 3. Configuration Reference

### Configuration Files

| File | Location | Purpose |
|------|----------|---------|
| Config | `~/.aws/config` | Profiles, region, output, plugins, service-specific settings |
| Credentials | `~/.aws/credentials` | Access keys and session tokens |
| Aliases | `~/.aws/cli/alias` | Command aliases (shorthand and external) |
| Credential Cache | `~/.aws/cli/cache/` | Cached assume-role credentials (JSON files) |

### Environment Variables

| Variable | Purpose | Overrides |
|----------|---------|-----------|
| `AWS_ACCESS_KEY_ID` | Access key for authentication | credentials file |
| `AWS_SECRET_ACCESS_KEY` | Secret key for authentication | credentials file |
| `AWS_SESSION_TOKEN` | Temporary session token | credentials file |
| `AWS_DEFAULT_REGION` | Default region for requests | config file `region` |
| `AWS_DEFAULT_OUTPUT` | Output format (json/text/table) | config file `output` |
| `AWS_PROFILE` | Named profile to use | `--profile` flag |
| `AWS_CONFIG_FILE` | Path to config file | default `~/.aws/config` |
| `AWS_SHARED_CREDENTIALS_FILE` | Path to credentials file | default `~/.aws/credentials` |
| `AWS_DATA_PATH` | Additional directories for service model data | Built-in models |
| `AWS_CA_BUNDLE` | Path to custom CA certificate bundle | System CA |
| `AWS_PAGER` | Pager program for output | system pager |
| `HTTPS_PROXY` / `HTTP_PROXY` | Proxy configuration | Direct connection |
| `NO_PROXY` | Hosts to bypass proxy | None |

### Config File Options (`~/.aws/config`)

```ini
[default]
region = us-east-1
output = json
cli_pager = less

[profile dev]
region = us-west-2
output = table
role_arn = arn:aws:iam::123456789012:role/dev-role
source_profile = default
credential_source = Environment

[plugins]
myplugin = awscli_plugin_module
```

| Option | Values | Default | Description |
|--------|--------|---------|-------------|
| `region` | Any AWS region code | None (required) | Default region for API calls |
| `output` | `json`, `text`, `table` | `json` | Output format |
| `cli_pager` | Pager command or empty string | System pager | Output pager; empty string disables |
| `role_arn` | IAM role ARN | None | Role to assume for this profile |
| `source_profile` | Profile name | None | Profile providing credentials for role assumption |
| `credential_source` | `Environment`, `Ec2InstanceMetadata`, `EcsContainer` | None | Where to find credentials for role assumption |
| `external_id` | String | None | External ID for cross-account role assumption |
| `mfa_serial` | MFA device ARN | None | MFA device for role assumption |
| `duration_seconds` | Integer | 3600 | Session duration for assumed roles |
| `ca_bundle` | File path | System CA | Custom CA certificate bundle |
| `parameter_validation` | `true`/`false` | `true` | Client-side parameter validation |

> [!TIP]
> Use `aws configure` to interactively set up profiles. Use `aws configure set` for scripted configuration. Use `aws configure list` to see which config values are active and where they came from.

### Alias Configuration (`~/.aws/cli/alias`)

```ini
[toplevel]
whoami = sts get-caller-identity
running-instances = ec2 describe-instances --filters Name=instance-state-name,Values=running
myip = !curl -s https://checkip.amazonaws.com
```

Two alias types exist:

| Type | Syntax | Implementation Class |
|------|--------|---------------------|
| Service alias | `name = service operation [args]` | `ServiceAliasCommand` |
| External alias | `name = !shell-command` | `ExternalAliasCommand` |

Service aliases are parsed and executed within the CLI process. External aliases (prefixed with `!`) spawn a subprocess.

---

## 4. Deployment & Scaling

### Installation Methods

**pip (recommended for v1):**
```bash
pip install awscli
```

**From source:**
```bash
git clone https://github.com/aws/aws-cli.git
cd aws-cli
pip install -e .
```

**System package managers:**
```bash
# Debian/Ubuntu
sudo apt install awscli

# RHEL/CentOS
sudo yum install awscli

# macOS (Homebrew)
brew install awscli@1
```

### Shell Completion Setup

**Bash:**
```bash
# Add to ~/.bashrc
complete -C aws_completer aws
# Or source the helper:
source /path/to/bin/aws_bash_completer
```

**Zsh:**
```bash
# Add to ~/.zshrc
source /path/to/bin/aws_zsh_completer.sh
```

The completer binary (`bin/aws_completer`) implements tab completion by introspecting botocore service models dynamically.

### Multi-Account / Multi-Region Deployment

> [!NOTE]
> AWS CLI v1 supports multiple named profiles natively. Each profile can target a different account, region, and credential source.

For fleet deployments:

1. **Shared credentials via instance profiles** - Use `credential_source = Ec2InstanceMetadata` or `EcsContainer` to avoid distributing static keys.
2. **Role chaining** - Use `role_arn` + `source_profile` to centralize credential management.
3. **Config distribution** - Distribute `~/.aws/config` via configuration management (Ansible, Chef, Puppet). Keep `~/.aws/credentials` empty when using instance roles.

### Containerized Usage

```dockerfile
FROM python:3.11-slim
RUN pip install awscli==1.44.85
# Credentials injected via environment or instance role
ENV AWS_DEFAULT_REGION=us-east-1
ENTRYPOINT ["aws"]
```

---

## 5. Operations

### Credential Resolution Order

The CLI resolves credentials in this fixed order (first match wins):

1. Command-line options (`--profile`)
2. Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`)
3. Credentials file (`~/.aws/credentials`)
4. Config file (`~/.aws/config`)
5. Assume-role (if `role_arn` configured, using source credentials)
6. Container credentials (ECS task role via `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`)
7. Instance metadata (EC2 instance profile via IMDS)

### Credential Caching

When using `role_arn` (assume-role), the CLI caches temporary credentials in `~/.aws/cli/cache/` as JSON files. The cache is implemented by `JSONFileCache` and keyed by a hash of the role parameters. Cached credentials are reused until they expire.

### Output Formats

| Format | Flag | Use Case |
|--------|------|----------|
| JSON | `--output json` | Machine parsing, jq pipelines |
| Text | `--output text` | Tab-separated, grep/awk friendly |
| Table | `--output table` | Human-readable aligned tables |

All formatters support `--query` (JMESPath) for server-side result filtering.

### Pagination

The CLI auto-paginates by default. Control with:

| Option | Effect |
|--------|--------|
| `--no-paginate` | Return only first page |
| `--page-size N` | Items per API call (reduces throttling) |
| `--max-items N` | Total items returned |
| `--starting-token` | Resume from a previous `NextToken` |

### Retry Behavior

Retries are handled by botocore with exponential backoff. The default retry mode retries up to 4 times for throttling and transient errors. Configure via:

```ini
[default]
retry_mode = standard  # or adaptive, legacy
max_attempts = 5
```

> [!WARNING]
> Setting `max_attempts` too high with `adaptive` mode can cause long delays under sustained throttling. Monitor CloudWatch API call metrics to tune appropriately.

### Logging and Debugging

```bash
# Enable debug output (sent to stderr)
aws --debug s3 ls

# Log botocore wire-level details
aws --debug ec2 describe-instances 2> debug.log
```

Debug output includes: credential resolution, endpoint calculation, request signing, wire-format request/response, retry attempts, and event handler invocations.

### History (Command Logging)

The `aws history` customization (`awscli/customizations/history/`) records CLI invocations:

```bash
aws history show       # Show recent commands
aws history list       # List recorded command IDs
```

---

## 6. Performance

### S3 Transfer Configuration

The `aws s3` commands (cp, sync, mv) use `s3transfer` for high-performance multipart operations:

| Config | Default | Description |
|--------|---------|-------------|
| `max_concurrent_requests` | 10 | Parallel transfer threads |
| `multipart_threshold` | 8 MB | File size above which multipart is used |
| `multipart_chunksize` | 8 MB | Size of each part |
| `max_queue_size` | 1000 | Maximum queued transfer operations |
| `max_bandwidth` | None | Bytes/sec bandwidth limit |

Configure in `~/.aws/config`:
```ini
[default]
s3 =
  max_concurrent_requests = 20
  multipart_threshold = 64MB
  multipart_chunksize = 16MB
  max_bandwidth = 50MB/s
```

### S3 Benchmarking

The `scripts/performance/benchmark_utils.py` module provides utilities for measuring S3 transfer performance. It supports:

- Configurable file sizes and concurrency levels
- Throughput measurement (MB/s)
- Comparison across different `s3transfer` configurations

### CLI Startup Time

CLI startup involves loading botocore, parsing config, loading plugins, and building the command table. Strategies to minimize cold-start latency:

- Avoid unnecessary plugins in `[plugins]`
- Use `AWS_DATA_PATH` only when custom models are needed
- On Lambda/containers, pre-warm by running `aws --version` in init

> [!TIP]
> If you only use a few services, startup time is not significantly affected because command table construction is lazy - individual service models load only when that service is invoked.

---

## 7. Security

### Credential Security Best Practices

| Practice | Implementation |
|----------|---------------|
| Never hardcode credentials | Use IAM roles, instance profiles, or environment variables |
| Rotate access keys regularly | Use `aws iam create-access-key` + `aws iam delete-access-key` |
| Use MFA for sensitive operations | Configure `mfa_serial` in profile |
| Least-privilege IAM policies | Scope policies to required actions/resources |
| Audit credential usage | Enable CloudTrail, review with `aws cloudtrail lookup-events` |

### File Permissions

```bash
chmod 600 ~/.aws/credentials
chmod 600 ~/.aws/config
chmod 700 ~/.aws/cli/cache/
```

> [!CAUTION]
> The credentials file and cached role credentials contain sensitive material. Ensure they are readable only by the owning user. The CLI does NOT enforce file permissions automatically.

### Request Signing

All API requests are signed using AWS Signature Version 4 (handled by botocore). The signing process:

1. Creates a canonical request (method, path, query, headers, payload hash)
2. Creates a string to sign (algorithm, timestamp, credential scope, canonical request hash)
3. Derives a signing key from the secret access key + date + region + service
4. Computes HMAC-SHA256 signature
5. Adds `Authorization` header with credential, signed headers, and signature

### SSL/TLS

- All requests use HTTPS by default
- Custom CA bundles: `--ca-bundle` flag or `AWS_CA_BUNDLE` env var or `ca_bundle` config
- Certificate verification can be disabled with `--no-verify-ssl` (NOT recommended for production)

> [!CAUTION]
> Disabling SSL verification (`--no-verify-ssl`) exposes credentials and data to man-in-the-middle attacks. Only use for debugging with self-signed certificates in development environments.

### CloudFront URL Signing

The `awscli/customizations/cloudfront.py` module uses the `rsa` dependency to generate signed URLs and signed cookies for CloudFront distributions. Private keys must be PEM-encoded RSA keys associated with a CloudFront key pair.

### Paramfile Security

The CLI supports loading parameter values from files (`file://path`) and URLs (`https://url`). This is implemented in `awscli/paramfile.py`:

- `file://` - reads local file contents as the parameter value
- `fileb://` - reads local file as binary (base64-encoded for JSON params)
- `https://` - fetches URL contents (use with caution - validates SSL by default)

> [!WARNING]
> The `https://` paramfile scheme fetches content from arbitrary URLs at request time. Ensure URLs are trusted and served over verified TLS to prevent parameter injection.

---

## 8. Developer Internals

### Module Reference

| Module | File | Responsibility |
|--------|------|---------------|
| `clidriver` | `awscli/clidriver.py` | Main orchestrator: CLIDriver, create_clidriver(), command dispatch |
| `argparser` | `awscli/argparser.py` | MainArgParser, ServiceArgParser, ArgTableArgParser hierarchy |
| `arguments` | `awscli/arguments.py` | Argument type system: BaseCLIArgument, CLIArgument, CustomArgument, BooleanArgument, ListArgument |
| `commands` | `awscli/commands.py` | CLICommand, ServiceCommand, ServiceOperation dispatch |
| `argprocess` | `awscli/argprocess.py` | Argument processing and type coercion |
| `formatter` | `awscli/formatter.py` | JSON, Text, Table output formatting |
| `help` | `awscli/help.py` | Help page generation and rendering |
| `clidocs` | `awscli/clidocs.py` | CLIDocumentEventHandler and subclasses for doc generation |
| `paramfile` | `awscli/paramfile.py` | file://, fileb://, https:// parameter loading |
| `shorthand` | `awscli/shorthand.py` | ParamShorthandParser, ShorthandParser for compact input syntax |
| `plugin` | `awscli/plugin.py` | Plugin loading and initialization |
| `alias` | `awscli/alias.py` | AliasLoader, AliasCommandInjector, ServiceAliasCommand, ExternalAliasCommand |
| `utils` | `awscli/utils.py` | Shared utilities |
| `compat` | `awscli/compat.py` | Python version compatibility layer |
| `schema` | `awscli/schema.py` | JSON schema utilities for argument validation |
| `table` | `awscli/table.py` | Table rendering for `--output table` |
| `topictags` | `awscli/topictags.py` | Built-in help topic management |
| `handlers` | `awscli/handlers.py` | Central registration of all customization event handlers |
| `bcdoc/` | `awscli/bcdoc/*.py` | Document generation: ReSTDocument, TextWriter, docevents, style |
| `customizations/` | `awscli/customizations/*.py` | Per-service CLI enhancements |

### Command Dispatch Architecture

```
CLIDriver.main(args)
  │
  ├── parse top-level args (--region, --output, --profile, --debug)
  │
  ├── look up service in command_table
  │        command_table built from botocore.session.get_available_services()
  │        + customizations injected via 'building-command-table.main' event
  │
  ├── ServiceCommand.__call__(remaining_args, parsed_globals)
  │     │
  │     ├── build operation sub-command table from service model
  │     │     + customizations via 'building-command-table.<service>' event
  │     │
  │     └── ServiceOperation.__call__(remaining_args, parsed_globals)
  │           │
  │           ├── build argument table from operation model
  │           │     + customizations via 'building-argument-table' event
  │           │
  │           ├── parse operation-specific args
  │           │
  │           ├── fire 'process-cli-arg' for each argument
  │           │
  │           ├── invoke botocore API call
  │           │
  │           └── format + display response
  │
  └── return exit code (0=success, 1=client error, 2=server error, 255=other)
```

### Argument Type System

The argument hierarchy enables both model-driven and custom arguments:

```
BaseCLIArgument (abstract)
  ├── CLIArgument         ← from botocore operation model
  ├── CustomArgument      ← manually defined (customizations)
  │     ├── BooleanArgument
  │     └── ListArgument
  └── (per-customization subclasses)
```

Each argument knows how to:
- Add itself to an argparse parser (`add_to_parser()`)
- Process its parsed value into API-compatible format (`add_to_params()`)

### Shorthand Syntax Parser

The shorthand parser (`awscli/shorthand.py`) allows compact input without JSON:

```bash
# Instead of:
aws ec2 run-instances --tag-specifications '[{"ResourceType":"instance","Tags":[{"Key":"Name","Value":"test"}]}]'

# Shorthand:
aws ec2 run-instances --tag-specifications ResourceType=instance,Tags=[{Key=Name,Value=test}]
```

Parser implementation uses a recursive descent approach. Key error types:
- `ShorthandParseSyntaxError` - malformed shorthand expression
- `DuplicateKeyInObjectError` - repeated keys in shorthand object
- `ParamUnknownKeyError` - unrecognized key name

### Customization Registration Pattern

All customizations are registered centrally in `awscli/handlers.py`:

```python
# Pattern: register handler for a specific event
def register_custom_commands(event_handlers):
    event_handlers.register('building-command-table.main', add_custom_commands)

# Handler modifies the command table in-place
def add_custom_commands(command_table, session, **kwargs):
    command_table['my-command'] = MyCustomCommand(session)
```

Major customizations include:

| Customization | Module | Adds |
|---------------|--------|------|
| S3 high-level commands | `customizations/s3/` | `aws s3 cp`, `sync`, `mv`, `rm`, `ls`, `mb`, `rb`, `presign` |
| EMR cluster management | `customizations/emr/` | `aws emr create-cluster`, `add-steps`, `ssh`, `put`, `socks` |
| Lambda | `customizations/lambda_cmd.py` | Zip packaging for `update-function-code` |
| Configure | `customizations/configure/` | `aws configure` interactive setup |
| CloudFront signing | `customizations/cloudfront.py` | `aws cloudfront sign` |
| ECS | `customizations/ecs/` | Task definition helpers |
| EKS | `customizations/eks/` | Kubeconfig management |
| CodeDeploy | `customizations/codedeploy/` | Deployment push helpers |
| CloudTrail | `customizations/cloudtrail.py` | Log validation |
| Logs | `customizations/logs/` | `aws logs tail` |
| History | `customizations/history/` | `aws history show/list` |

### Help System Internals

Help generation uses the `bcdoc` subpackage:

1. `CLIDocumentEventHandler` (and variants: Provider, Service, Operation) listens for doc events
2. Generates reStructuredText from botocore model descriptions + `awscli/data/` overlays
3. On Unix: pipes reST through `rst2man.py` (docutils) then `groff`/`mandoc` to the pager
4. On Windows: renders plain text directly to the pager
5. `awscli/data/` contains per-service overlays adding CLI-specific examples to help pages

Document event flow:
```
doc-title → doc-description → doc-synopsis → doc-options → doc-examples → doc-output
```

Each event handler class (`ProviderDocumentEventHandler`, `ServiceDocumentEventHandler`, `OperationDocumentEventHandler`) generates appropriate content for its scope level.

### Plugin System

Plugins are loaded from the `[plugins]` section of `~/.aws/config`:

```ini
[plugins]
my_plugin = my_plugin_module
```

The plugin loading process (`awscli/plugin.py`):

1. Import the module specified as the value
2. Call `module.awscli_initialize(event_hooks)` passing the botocore event emitter
3. The plugin registers handlers for whatever events it needs

Plugins can:
- Add new top-level commands
- Modify argument tables
- Intercept and transform requests/responses
- Add new output formats
- Inject custom credential providers

### Data-Driven Model Architecture

The CLI is fundamentally data-driven via botocore service models:

- **Service models** (JSON): define every API operation, its inputs, outputs, and errors
- **CLI overlays** (`awscli/data/`): add CLI-specific metadata (examples, extra descriptions)
- **Custom data path** (`AWS_DATA_PATH`): allows injecting additional/override models

This means adding a new AWS service to the CLI requires zero code changes - only a new service model JSON file from botocore.

### Error Handling

The codebase defines 86 error types across modules:

| Category | Key Errors | Module |
|----------|-----------|--------|
| Argument parsing | `ParamError`, `ParamSyntaxError`, `ParamUnknownKeyError`, `TooComplexError` | `argprocess` |
| Shorthand | `ShorthandParseSyntaxError`, `DuplicateKeyInObjectError` | `shorthand` |
| Alias | `InvalidAliasException` | `alias` |
| CLI | `UnknownArgumentError` | `argparser` |
| Help rendering | `ExecutableNotFoundError` (groff/mandoc) | `help` |
| Paramfile | `ResourceLoadingError` | `paramfile` |
| EMR | 30+ specific error classes | `customizations/emr/` |

Exit codes: `0` = success, `1` = client error (4xx or argument error), `2` = server error (5xx), `255` = unhandled exception.

### Testing Strategy

> [!NOTE]
> The test suite uses pytest and is organized into three tiers with increasing scope and cost.

| Tier | Location | Strategy | Runs Against |
|------|----------|----------|-------------|
| Unit | `tests/unit/` | Heavy mocking, CLIRunner + SessionStubber | No network |
| Functional | `tests/functional/` | Real CLI invocation with stubbed HTTP responses | No network |
| Integration | `tests/integration/` | Real AWS API calls | Live AWS (requires credentials) |

**Key test infrastructure:**
- `CLIRunner` - test helper that invokes CLI commands programmatically
- `SessionStubber` - patches botocore HTTP layer to return canned responses
- Pre-commit hooks: `check-yaml`, `end-of-file-fixer`, `trailing-whitespace`, `isort`, `ruff`

Running tests:
```bash
# Unit tests
pytest tests/unit/

# Functional tests
pytest tests/functional/

# Specific test file
pytest tests/unit/test_clidriver.py

# With coverage
pytest --cov=awscli tests/unit/
```

### Contributing Patterns

When adding a new customization:

1. Create module in `awscli/customizations/`
2. Define command class extending `BasicCommand` (for new commands) or register event handlers (for modifications)
3. Register in `awscli/handlers.py` via the appropriate event
4. Add CLI overlay data in `awscli/data/` if needed (examples, descriptions)
5. Write unit tests in `tests/unit/customizations/`
6. Write functional tests in `tests/functional/`

---

## 9. Integration Points

### Botocore Integration

The CLI is a thin layer over botocore. Key integration points:

| CLI Component | Botocore Component | Integration |
|---------------|-------------------|-------------|
| CLIDriver | `botocore.session.Session` | Session provides service models, credentials, config |
| ServiceCommand | `botocore.service_model` | Operations derived from service model |
| ServiceOperation | `botocore.client` | Actual API calls via botocore client |
| Argument types | `botocore.model.Shape` | Argument metadata from model shapes |
| Events | `botocore.hooks.EventEmitter` | Shared event system between CLI and SDK |
| Credentials | `botocore.credentials` | Full credential provider chain |
| Retries | `botocore.retries` | Retry logic lives in botocore |
| Pagination | `botocore.paginate` | Paginator from service model pagination config |

### S3Transfer Integration

High-level S3 commands (`aws s3 cp/sync/mv`) use the `s3transfer` library for:

- Multipart uploads (parallel part uploading)
- Multipart downloads (parallel part downloading)
- Transfer progress callbacks
- Bandwidth throttling
- Automatic content-type detection

### Shell Integration

| Feature | Mechanism |
|---------|-----------|
| Tab completion | `bin/aws_completer` binary + bash/zsh completion scripts |
| Pager output | Respects `AWS_PAGER`, `PAGER`, or system default |
| External aliases | `!` prefix spawns subprocess with shell |
| Paramfile | `file://` reads filesystem; `fileb://` reads binary |
| Output piping | Stdout is the formatted response; stderr for errors/debug |

> [!IMPORTANT]
> When piping CLI output, the pager is automatically disabled. The CLI detects non-TTY stdout and writes directly to the pipe.

### CI/CD Integration

```bash
# Non-interactive usage pattern
export AWS_DEFAULT_REGION=us-east-1
export AWS_ACCESS_KEY_ID=$CI_AWS_KEY
export AWS_SECRET_ACCESS_KEY=$CI_AWS_SECRET

# Disable pager for scripted use
export AWS_PAGER=""

# Use --output text for easy parsing
INSTANCE_ID=$(aws ec2 run-instances \
  --image-id ami-12345 \
  --instance-type t3.micro \
  --query 'Instances[0].InstanceId' \
  --output text)
```

### Docker / Container Integration

When running in containers:

- **ECS tasks**: credentials auto-injected via `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`
- **EC2 instances**: IMDS provides credentials automatically
- **Kubernetes**: use IAM Roles for Service Accounts (IRSA) which sets `AWS_WEB_IDENTITY_TOKEN_FILE`

> [!TIP]
> In ephemeral containers, avoid writing to `~/.aws/` entirely. Use environment variables for credentials and pass `--region` explicitly. This eliminates filesystem state and simplifies container image management.

### Custom Data Path

The `AWS_DATA_PATH` environment variable allows injecting additional service models:

```bash
export AWS_DATA_PATH=/opt/custom-models

# Structure:
# /opt/custom-models/
#   myservice/
#     2024-01-01/
#       service-2.json    (service model)
#       paginators-1.json (pagination config)
#       waiters-2.json    (waiter definitions)
```

This enables using AWS CLI with custom/internal services that follow the botocore model format, or overriding behavior of existing services for testing.

---

## Quick Reference

### Common Administrative Commands

```bash
# Show active configuration and source
aws configure list

# Show caller identity (verify credentials)
aws sts get-caller-identity

# List all configured profiles
aws configure list-profiles

# Set a config value
aws configure set region us-west-2 --profile prod

# Get a config value
aws configure get region --profile prod

# Show CLI version
aws --version
```

### Troubleshooting Checklist

| Symptom | Likely Cause | Resolution |
|---------|-------------|-----------|
| `Unable to locate credentials` | No valid credential source | Check env vars, credentials file, instance role |
| `ExpiredTokenException` | Cached assume-role token expired | Delete `~/.aws/cli/cache/` contents |
| `Could not connect to the endpoint URL` | Wrong region or network issue | Verify region, check proxy/firewall |
| `An error occurred (AccessDenied)` | Insufficient IAM permissions | Review IAM policy for the identity |
| `command not found: aws` | Not in PATH | Verify installation, check `which aws` |
| Help shows "groff" error | groff/mandoc not installed | Install groff or use `aws help 2>/dev/null \| cat` |
| Slow startup | Many plugins or large custom data path | Audit `[plugins]`, check `AWS_DATA_PATH` |

---

_Generated May 2026 from aws/aws-cli at f141b6c. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._