# 🐸 Coqui TTS Admin & Developer Guide

This guide is for the engineers, ops teams, and architects who **deploy, operate, extend, or train with** Coqui TTS. It covers the system architecture, the model and config systems, deployment topologies, the training pipeline, security posture, and the internals needed to maintain or fork the project. End-user synthesis instructions live in the companion **User Guide**.

> [!IMPORTANT]
> This documentation describes Coqui TTS **v0.22.0** at commit `dbf1a08` (2024-02-10). The company that built Coqui (Coqui GmbH) has **shut down**, so this codebase is effectively frozen at this release and is now community-maintained. There will be no upstream patches for new PyTorch/`transformers` incompatibilities or security findings unless contributed by the community. Treat any deployment as running on **unmaintained, archived software** and plan dependency pinning and security mitigations accordingly.

---

## 1. Executive Summary

Coqui TTS is a PyTorch-based deep-learning toolkit for text-to-speech. It packages:

- **A model zoo** — ~70 pretrained TTS models, ~17 vocoders, and a voice-conversion model, registered in a single `TTS/.models.json` manifest, plus on-demand access to ~1100 Meta MMS/Fairseq voices.
- **Three runtime surfaces** — a `tts` CLI (`TTS/bin/synthesize.py`), a Python API (`TTS/api.py`), and a Flask demo server (`TTS/server/server.py`, console script `tts-server`).
- **A training stack** — per-architecture model classes under `TTS/tts`, `TTS/vocoder`, `TTS/vc`, `TTS/encoder`, driven by the external `trainer` package and `coqpit` dataclass configs, with ready-made recipes under `recipes/`.

| Attribute | Value |
|---|---|
| Version | 0.22.0 |
| Language | Python (≥3.9, <3.12) |
| License (code) | MPL-2.0 |
| Core runtime | PyTorch ≥2.1, torchaudio |
| Config system | `coqpit` dataclasses (per-model `config.json`) |
| Training engine | external `trainer` package |
| Web layer | Flask (demo server) |
| Author | Eren Gölge / Coqui |
| Source commit | `dbf1a08` |

> [!NOTE]
> The **code** is MPL-2.0, but **individual models carry their own licenses** spanning Apache-2.0, MPL, MIT, CC-BY-SA, CC-BY-NC-ND, and the bespoke **CPML** (XTTS — non-commercial). Licensing is per-model, not project-wide. See [§5.4](#54-model-licensing).

---

## 2. Architecture

### 2.1 High-level component map

```text
                         ┌─────────────────────────────────────────────┐
   text ──▶  Entry surface │  tts (CLI)   tts-server (Flask)   TTS.api   │
                         └───────────────────────┬─────────────────────┘
                                                 │
                                       ┌─────────▼──────────┐
                                       │   Synthesizer       │  TTS/utils/synthesizer.py
                                       │  (orchestrator)     │
                                       └───┬───────┬─────┬───┘
                          ┌────────────────┘       │     └───────────────┐
                  ┌───────▼────────┐      ┌────────▼───────┐    ┌─────────▼────────┐
                  │  tts_model      │      │  vocoder model │    │ speaker encoder  │
                  │ TTS/tts/models  │      │ TTS/vocoder    │    │ TTS/encoder      │
                  └───────┬─────────┘      └────────────────┘    └──────────────────┘
                          │
                 ┌────────▼─────────┐   ┌───────────────────┐
                 │  ModelManager     │   │ load_config        │
                 │ TTS/utils/manage  │   │ TTS/config         │  (coqpit)
                 │ download + cache  │   └───────────────────┘
                 └────────┬──────────┘
                          ▼
                  TTS/.models.json  (model registry) ──▶ remote model archives (HF / GitHub releases)
```

The **Synthesizer** (`TTS/utils/synthesizer.py`) is the orchestration hub. It loads a TTS model, an optional vocoder, and an optional speaker encoder, then exposes `.tts()` and `.voice_conversion()`. Both the CLI and the server construct a `Synthesizer`; the `TTS.api.TTS` class wraps it with a friendlier interface.

### 2.2 Package layout

| Path | Responsibility |
|---|---|
| `TTS/api.py` | High-level Python API (`TTS` class). |
| `TTS/bin/` | Executables: `synthesize.py` (the `tts` CLI), `train_tts.py`, `train_vocoder.py`, `train_encoder.py`, dataset/compute utilities. |
| `TTS/tts/` | Text-to-speech models (`models/`), neural layers (`layers/`), configs (`configs/`), datasets, and text/phoneme utilities (`utils/text/`). |
| `TTS/vocoder/` | Vocoder models (MelGAN, HiFiGAN, WaveGrad, WaveRNN, UnivNet, …), layers, datasets. |
| `TTS/vc/` | Voice-conversion models (FreeVC). |
| `TTS/encoder/` | Speaker/voice encoder models (GE2E, ResNet, angular-loss). |
| `TTS/utils/` | `ModelManager` (`manage.py`), `Synthesizer` (`synthesizer.py`), audio processing (`audio/`), downloads (`download.py`), generic utilities. |
| `TTS/config/` | `load_config()` and base config classes (coqpit). |
| `TTS/server/` | Flask demo server + HTML templates. |
| `TTS/model.py` | `BaseTrainerModel` — the abstract base every model inherits. |
| `recipes/` | End-to-end training recipes per dataset (ljspeech, vctk, thorsten_DE, kokoro, …). |
| `notebooks/` | Jupyter notebooks for dataset analysis and evaluation. |
| `docs/` | Sphinx documentation sources. |
| `tests/` | Unit, data, inference, and zoo (model-download) tests. |

### 2.3 The model abstraction

Every model inherits `BaseTrainerModel` (`TTS/model.py`), which extends `TrainerModel` from the external `trainer` package and requires three methods:

```python
class BaseTrainerModel(TrainerModel):
    @staticmethod
    @abstractmethod
    def init_from_config(config: Coqpit): ...   # build the model from its config

    @abstractmethod
    def inference(self, input, aux_input={}) -> Dict:  # returns {"model_outputs": ...}

    @abstractmethod
    def load_checkpoint(self, config, checkpoint_path, eval=False, strict=True, cache=False): ...
```

This uniform contract is why one `Synthesizer` can drive Tacotron2, VITS, XTTS, or a vocoder interchangeably — they all expose `init_from_config` / `inference` / `load_checkpoint`.

### 2.4 Model families

Coqui implements a broad set of architectures. Knowing which family a model belongs to tells you whether it needs a separate vocoder and what its inference cost looks like.

| Family | Type | Models | Needs vocoder? |
|---|---|---|---|
| **Spectrogram (two-stage)** | acoustic | Tacotron, Tacotron2, Glow-TTS, Speedy-Speech, Align-TTS, FastPitch, FastSpeech/2, SC-GlowTTS, Capacitron, OverFlow, Neural HMM, Delightful TTS | Yes |
| **End-to-end** | full | VITS, YourTTS, ⓍTTS (XTTS), Tortoise, Bark | No |
| **Vocoders** | waveform | MelGAN, Multiband-MelGAN, ParallelWaveGAN, GAN-TTS discriminators, WaveRNN, WaveGrad, HiFiGAN, UnivNet | — |
| **Speaker encoders** | embedding | GE2E, Angular-loss, ResNet | — |
| **Voice conversion** | conversion | FreeVC | — |

> [!NOTE]
> Two-stage models output a mel-spectrogram and are paired with a vocoder (each model declares a `default_vocoder` in `.models.json`). If no vocoder is supplied, the Synthesizer falls back to **Griffin-Lim**, which is lower quality. End-to-end models (VITS/XTTS) skip the vocoder entirely.

---

## 3. The Model Registry & Manager

### 3.1 `.models.json`

`TTS/.models.json` is the canonical registry. It is a nested tree keyed by `model_type → language → dataset → model_name`. Each leaf describes one model:

```json
"xtts_v2": {
  "description": "XTTS-v2.0.3 by Coqui with 17 languages.",
  "hf_url": ["https://.../model.pth", "https://.../config.json", "..."],
  "model_hash": "10f92b55c512af7a8d39d650547a15a7",
  "default_vocoder": null,
  "license": "CPML",
  "tos_required": true
}
```

Key fields:

| Field | Meaning |
|---|---|
| `hf_url` / `github_rls_url` | Download source(s). A list means a multi-file model directory. |
| `model_hash` / `model_hash` | MD5 checksum verified after download. |
| `default_vocoder` | Vocoder name to pair with (or `null` for end-to-end). |
| `license` | Per-model license string. |
| `tos_required` | If `true`, the user must accept terms before download (XTTS). |
| `commit`, `author`, `contact` | Provenance metadata. |

### 3.2 `ModelManager` (`TTS/utils/manage.py`)

`ModelManager` reads `.models.json`, resolves a model name to a download, fetches and verifies archives, and caches them. Responsibilities:

- **`list_models()`** — enumerate the registry (backs `tts --list_models`).
- **`download_model(name)`** — resolve, download, extract, and return local paths.
- **`model_info_by_idx` / `model_info_by_full_name`** — registry lookups.
- **Terms-of-service gate** — raises `Exception: [!] You must agree to the terms of service to use this model.` for `tos_required` models unless `COQUI_TOS_AGREED=1`.
- **Hash verification** — `TTS/utils/download.py` raises `RuntimeError: The hash of {file} does not match.` on a checksum mismatch.

### 3.3 Cache resolution

The model cache directory is resolved by `get_user_data_dir()` in `TTS/utils/generic_utils.py`:

| Priority | Source | Result |
|---|---|---|
| 1 | `TTS_HOME` env var | `$TTS_HOME/tts/` |
| 2 | `XDG_DATA_HOME` env var | `$XDG_DATA_HOME/tts/` |
| 3 | default | `~/.local/share/tts/` |

> [!TIP]
> In containers and CI, set `TTS_HOME` to a mounted volume so model downloads (which can be hundreds of MB to several GB for XTTS/Bark/Tortoise) persist across runs and don't bloat the image layer or re-download on every start.

---

## 4. Configuration

Coqui has **almost no global configuration** — there is no central YAML or `.env` for the library itself. Behavior is driven two ways:

1. **Per-model config** — each model ships a `config.json` parsed into a `coqpit` dataclass via `TTS/config/load_config()`. This holds audio settings (sample rate, mel parameters), model hyperparameters, dataset definitions, and inference options. When you run a released model, its config travels with the download; when you run your own, you pass `--config_path`.
2. **Environment variables** — the only three runtime env vars are below.

| Variable | Read in | Purpose |
|---|---|---|
| `TTS_HOME` | `generic_utils.py` | Model cache base directory (priority 1). |
| `XDG_DATA_HOME` | `generic_utils.py` | Model cache base directory (priority 2). |
| `COQUI_TOS_AGREED` | `manage.py` | Set to `1` to pre-accept terms for gated models. |

### 4.1 Working with a model config

```python
from TTS.config import load_config

config = load_config("path/to/config.json")
print(config.audio.sample_rate)
print(config.model)            # architecture name
```

`load_config()` raises:
- `ModuleNotFoundError: [!] Config for {model_name} cannot be found.` — unknown model config.
- `TypeError: [!] Unknown config file type {ext}` — config file is neither `.json` nor a supported type.

> [!NOTE]
> Because config is a `coqpit` dataclass, the **schema is the Python class** (e.g. `Tacotron2Config`, `VitsConfig`, `XttsConfig` under `TTS/tts/configs/`), not a documented key list. To see every tunable for a model, read its `*Config` dataclass. This is the authoritative config reference for training and custom inference.

---

## 5. Security

> [!CAUTION]
> Coqui TTS has several material security considerations. Because the project is unmaintained, **you own these mitigations** — there will be no upstream fix. Review this section before any production or multi-user deployment.

### 5.1 Unsafe model deserialization (pickle)

Model checkpoints are PyTorch `.pth` files loaded via `torch.load()` (through `load_fsspec`), which uses Python's `pickle` and can **execute arbitrary code** during deserialization. Combined with on-demand downloads from remote URLs, a tampered checkpoint is a remote-code-execution vector.

- **Mitigation:** Only load checkpoints from sources you trust. Verify the `.models.json` checksums (Coqui does this for registry models). For custom or third-party checkpoints, scan them and load in an isolated environment. Where possible on newer PyTorch, prefer `weights_only=True` loading semantics.
- **Reference:** community issue #4409 (Unsafe Deserialization of Model Checkpoints).

### 5.2 Archive extraction path traversal

Model archives are unpacked with `extractall()` in `TTS/utils/download.py` without sanitizing member paths, so a malicious archive could write files outside the cache directory (Zip-Slip).

- **Mitigation:** Only download models from the official registry/URLs. Run the process as a non-root user with a constrained `TTS_HOME` so any traversal is contained.
- **Reference:** community issue #4409 (Path Traversal in Archive Extraction). `download.py` only accepts `.tar.gz`, `.tgz`, and `.zip` and raises `NotImplementedError` for other formats — but does not guard member paths.

### 5.3 Demo server exposure

The Flask server (`TTS/server/server.py`) is for local experimentation:

- It binds to **`host="::"`** (all interfaces) in `app.run()`.
- It has **no authentication, rate limiting, or CSRF protection**.
- `--debug true` enables Flask's debugger (Werkzeug), which must **never** be exposed publicly.
- `/api/tts` synthesizes arbitrary attacker-supplied text; a single global `Lock` serializes requests, so it is also trivially DoS-able by long inputs.

> [!WARNING]
> Never expose `tts-server` directly to untrusted networks. Front it with an authenticated reverse proxy, bind it to `localhost`, disable `--debug`, and cap request sizes. Treat it as a developer convenience, not a production service.

### 5.4 Model licensing

Licensing is **per-model** and some models are non-commercial:

| License | Examples | Commercial use |
|---|---|---|
| Apache-2.0 / MIT / MPL | most Tacotron2, VITS, HiFiGAN voices | Generally yes (verify) |
| CC-BY-SA 4.0 | openbible voices, some community voices | Share-alike conditions |
| CC-BY-NC-ND 4.0 | YourTTS, some VCTK | **No** (non-commercial) |
| **CPML** | **XTTS v1.1 / v2** | **No** (non-commercial) |

> [!IMPORTANT]
> Always read the `license` field from `.models.json` (or `tts --model_info_by_name`) before shipping a product that uses a Coqui voice. XTTS — the flagship model — is **non-commercial CPML** and is gated behind a terms-of-service acceptance (`tos_required: true`).

### 5.5 Defunct domain

The original `coqui.ai` domain is no longer controlled by the project after the company's shutdown (community issues #4418, #4392 report it as hijacked). Do not fetch models, configs, or scripts from `coqui.ai`; rely on the GitHub repository and the registry URLs (which point at `coqui.gateway.scarf.sh` and Hugging Face mirrors).

---

## 6. Deployment

### 6.1 Topologies

| Topology | How | When |
|---|---|---|
| **Library embed** | `pip install TTS`; call `TTS.api.TTS` in-process | Batch synthesis, pipelines, apps. |
| **CLI / batch** | `tts ...` invoked per job | Scripted/offline generation. |
| **Demo server** | `tts-server --model_name ...` behind a proxy | Interactive internal tooling. |
| **Docker** | `ghcr.io/coqui-ai/tts-cpu` / GPU images | Reproducible, isolated runs. |

### 6.2 Docker

```bash
docker run --rm -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/coqui-ai/tts-cpu
python3 TTS/server/server.py --model_name tts_models/en/vctk/vits
```

GPU variants are published alongside the CPU image. Mount a volume at `TTS_HOME` to persist the model cache.

### 6.3 Deployment checklist

- [ ] Pin the Python interpreter to **3.9–3.11** (the package refuses 3.12+).
- [ ] Pin `transformers` to the **4.x** line (5.x breaks XTTS — see §8).
- [ ] Pin `setuptools` to a version that still ships `pkg_resources`.
- [ ] Pre-download models into a persistent `TTS_HOME` volume.
- [ ] Set `COQUI_TOS_AGREED=1` if (and only if) the model license permits your use.
- [ ] Confirm each model's license fits your distribution model.
- [ ] If serving HTTP: bind to localhost, add auth, disable `--debug`, cap input length.
- [ ] Run as a non-root user with a constrained cache directory.
- [ ] Provision a GPU for XTTS/Bark/Tortoise workloads.

> [!TIP]
> Capture a known-good environment with `python -m TTS.bin.collect_env_info` and store it next to your deployment manifests. Given the unmaintained dependency surface, an exact, reproducible set of versions is your most important operational artifact.

---

## 7. Training & Operations

### 7.1 Training entry points (`TTS/bin/`)

| Script | Purpose |
|---|---|
| `train_tts.py` | Train a TTS (acoustic / end-to-end) model. |
| `train_vocoder.py` | Train a vocoder. |
| `train_encoder.py` | Train a speaker encoder. |
| `tune_wavegrad.py` | Tune WaveGrad noise schedules. |
| `extract_tts_spectrograms.py` | Pre-extract spectrograms for two-stage training. |
| `compute_embeddings.py` | Compute speaker d-vector embeddings. |
| `compute_statistics.py` | Compute dataset audio statistics (normalization). |
| `compute_attention_masks.py` | Precompute attention masks. |
| `resample.py` | Resample a dataset to a target sample rate. |
| `remove_silence_using_vad.py` | Trim leading/trailing silence with VAD. |
| `find_unique_chars.py` / `find_unique_phonemes.py` | Build the character/phoneme set for a dataset. |
| `eval_encoder.py` | Evaluate a trained speaker encoder. |
| `collect_env_info.py` | Dump environment details for bug reports. |

### 7.2 Recipes

`recipes/<dataset>/<model>/` holds ready-to-run training scripts (e.g. `recipes/ljspeech/vits_tts`, `recipes/vctk/yourtts`, `recipes/thorsten_DE/...`). A recipe wires a dataset config to a model config and calls the `trainer`. Start from the recipe closest to your data and adjust the config dataclass.

```bash
# Example shape of a recipe run
python recipes/ljspeech/vits_tts/train_vits.py
```

### 7.3 The training engine

Training is delegated to the external **`trainer`** package (`trainer>=0.0.36`). Models implement the `BaseTrainerModel` contract; `trainer` handles the loop, logging (terminal + TensorBoard), checkpointing, and distributed runs. Configs are `coqpit` dataclasses.

> [!CAUTION]
> **XTTS cannot be trained through the generic path.** `TTS/tts/models/xtts.py` raises `NotImplementedError: XTTS has a dedicated trainer ...` from its training hooks. Use the dedicated XTTS fine-tuning recipe/demo (`TTS/demos/xtts_ft_demo/`) instead. Likewise, **Tortoise training is not implemented** (`NotImplementedError: 'Tortoise Training is not implemented'`).

### 7.4 Day-to-day operations

| Task | How |
|---|---|
| Inspect available voices | `tts --list_models` / `--model_info_by_name` |
| Clear/refresh a model | delete its folder under `$TTS_HOME/tts/` and re-run |
| Monitor a training run | terminal logs + TensorBoard (written by `trainer`) |
| Serve a voice | `tts-server --model_name <name> --port 5002` |
| Capture environment | `python -m TTS.bin.collect_env_info` |

---

## 8. Known Issues & Compatibility

The most common production problems stem from dependency drift, because the codebase is frozen but its dependencies move.

| Symptom | Cause | Fix |
|---|---|---|
| `ImportError: cannot import name 'isin_mps_friendly' from 'transformers.pytorch_utils'` | `transformers` too new | Pin `transformers>=4.33,<5` (#4406, #4420). |
| `AttributeError: 'GenerationConfig' object has no attribute 'do_stream'` | `transformers` 5.0 breaks XTTS streaming | Pin `transformers<5` (#4394). |
| `ModuleNotFoundError: No module named 'pkg_resources'` | new `setuptools` removed `pkg_resources` | Install a `setuptools` that still provides it (#4410). |
| Install fails on Python 3.12+ | hard gate in `setup.py` | Use Python 3.9–3.11 (#4416 requests 3.12+ support). |
| XTTS errors inside Docker | environment/transformers mismatch | Use the published image + pinned deps (#4415). |
| First word skipped in output | attention artifact in some models | Add leading punctuation or switch to VITS/XTTS (#4016). |
| `gpu=True` warning | constructor arg deprecated | Use `.to(device)` instead. |

> [!NOTE]
> Feature requests that remain open against the frozen codebase include FunASR annotation support (#4422) and a standalone compiled binary (#4419). These will not land without community forks.

---

## 9. Code Quality & Conventions

- **Uniform model contract.** Every model implements `init_from_config` / `inference` / `load_checkpoint`, which is what makes the Synthesizer polymorphic across architectures.
- **Config-as-dataclass.** `coqpit` dataclasses give typed, serializable configs; the config *is* the schema. This keeps training reproducible but means there is no separate documented config reference.
- **Registry-driven distribution.** `.models.json` + `ModelManager` cleanly separate "what models exist" from "how to fetch them," and include checksums and license/ToS metadata.
- **Late imports for CLI speed.** `synthesize.py` imports `TTS.api`/`Synthesizer`/`ModelManager` *inside* `main()` so `tts --list_models` and help screens start fast.
- **Cython hot path.** `TTS/tts/utils/monotonic_align/core.pyx` is compiled for alignment performance — note the build dependency (`cython`) when packaging.
- **Broad text-frontend support.** Per-language G2P/phonemizers (`gruut`, `jieba`/`pypinyin`, `jamo`/`g2pkk`, `bangla`/`bnnumerizer`) live under `TTS/tts/utils/text/`.

### Improvement opportunities (for forks)

1. **Harden checkpoint loading** — adopt `weights_only=True` and reject untrusted pickles (§5.1).
2. **Sanitize archive extraction** — validate member paths before `extractall()` (§5.2).
3. **Modernize dependency pins** — support `transformers` 5.x and Python 3.12+ (#4394, #4416).
4. **Secure the server** — optional auth, request-size limits, and a `localhost` default bind (§5.3).
5. **Document config dataclasses** — auto-generate a config reference from the `*Config` classes.

---

## 10. Quick Reference

### 10.1 Console scripts

| Command | Entry point |
|---|---|
| `tts` | `TTS.bin.synthesize:main` |
| `tts-server` | `TTS.server.server:main` |

### 10.2 HTTP API (demo server)

| Method | Path | Purpose |
|---|---|---|
| GET | `/` | Web UI |
| GET | `/details` | Model details (`--show_details`) |
| GET/POST | `/api/tts` | Synthesize → `audio/wav` |
| GET | `/locales` | MaryTTS locales |
| GET | `/voices` | MaryTTS voices |
| GET/POST | `/process` | MaryTTS synthesis |

### 10.3 Environment variables

| Variable | Purpose |
|---|---|
| `TTS_HOME` | Model cache base (priority 1). |
| `XDG_DATA_HOME` | Model cache base (priority 2). |
| `COQUI_TOS_AGREED` | `1` to pre-accept gated-model terms. |

### 10.4 Key dependencies

| Dependency | Role |
|---|---|
| `torch>=2.1`, `torchaudio` | Model runtime. |
| `numpy`, `scipy`, `librosa`, `soundfile` | Audio/DSP. |
| `transformers>=4.33.0` | XTTS / Tortoise backbones. |
| `encodec`, `einops` | Bark / Tortoise. |
| `trainer>=0.0.36` | Training loop. |
| `coqpit>=0.0.16` | Config dataclasses. |
| `gruut[de,es,fr]`, `pysbd`, `jieba`, `pypinyin`, `jamo`, `g2pkk`, `bangla` | Text frontends / G2P. |
| `flask>=2.0.1` | Demo server. |
| `cython` | Monotonic-align extension. |

### 10.5 Critical operational facts

- Python **3.9–3.11 only**; `transformers` **4.x only**.
- Model cache at `$TTS_HOME/tts` → `$XDG_DATA_HOME/tts` → `~/.local/share/tts`.
- XTTS is **CPML (non-commercial)** and **gated** (`COQUI_TOS_AGREED=1`).
- Demo server has **no auth** and binds all interfaces — never expose it directly.
- Checkpoints are **pickle**; archives are extracted **without path guards** — load only trusted sources.
- The project is **unmaintained** — you own all patches and mitigations.

---

_Generated June 2026 from coqui-ai/TTS at dbf1a08. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._