# MiroFish Administrator & Developer Guide

This guide is the operational and architectural reference for **MiroFish** — a multi-agent swarm-intelligence prediction engine. It is written for the people who deploy, operate, extend, or audit the system: administrators, DevOps engineers, backend developers, and architects. End-user instructions live in the companion **User Guide**.

MiroFish turns unstructured seed documents into large-scale social simulations. A Vue 3 single-page app drives a Flask backend, which orchestrates an LLM (via the OpenAI SDK), a [Zep Cloud](https://www.getzep.com/) knowledge graph, and the [OASIS](https://github.com/camel-ai/oasis) multi-agent social-simulation framework. There is no database server and no message broker — state is persisted as JSON and CSV files on disk, and long-running work is handled by background threads plus subprocesses.

---

## 1. Executive Summary

| Aspect | Detail |
|--------|--------|
| **Purpose** | Generate prediction reports by simulating thousands of LLM-backed social agents seeded from documents |
| **Frontend** | Vue 3.5 SPA, Vite 7, Vue Router 4, vue-i18n 11, D3 7, axios; dev server on port **3000** |
| **Backend** | Flask 3 + flask-cors, OpenAI SDK, zep-cloud 3.13, camel-oasis 0.2.5 / camel-ai 0.2.78, PyMuPDF, pydantic; serves on port **5001** |
| **Runtime** | Node ≥ 18; Python ≥ 3.11, < 3.13; `uv` for Python deps |
| **External services** | An OpenAI-compatible LLM endpoint; Zep Cloud |
| **Persistence** | Filesystem only — JSON/CSV under `backend/uploads/` (projects, simulations, reports) |
| **Concurrency model** | Background `threading.Thread` for async API tasks; OASIS simulations run as **subprocesses** with IPC |
| **Auth** | **None.** No authentication or authorization layer exists (see [Security](#9-security-analysis)) |
| **License** | AGPL-3.0 |
| **Source commit documented** | `96096ea` (2026-05-25) |

> [!IMPORTANT]
> MiroFish as shipped is a **single-tenant, unauthenticated developer/research tool**. CORS is wide open (`origins: *` on `/api/*`), there is no login, and the Flask dev server is the runtime even in the Docker image. Treat any deployment as trusted-network-only unless you add an auth and reverse-proxy layer yourself.

---

## 2. Architecture

### 2.1 System diagram

```text
┌──────────────────────────────────────────────────────────────────┐
│                       Frontend — Vue 3 SPA                         │
│  Vite dev server :3000  ·  Vue Router  ·  vue-i18n  ·  D3 ·  axios │
│  Views: Home → MainView(Process) → SimulationView →                │
│         SimulationRunView → ReportView → InteractionView           │
└───────────────────────────┬──────────────────────────────────────┘
                            │ HTTP/JSON (axios; 5-min timeout; auto-retry)
                            │ dev: Vite proxies /api → :5001
                            ▼
┌──────────────────────────────────────────────────────────────────┐
│                      Backend — Flask :5001                         │
│  CORS(*) on /api/*   ·   before/after request logging              │
│                                                                    │
│  Blueprints:                                                       │
│    /api/graph/*       → graph_bp        (ontology, graph build)    │
│    /api/simulation/*  → simulation_bp   (entities, prepare, run)   │
│    /api/report/*      → report_bp       (report gen, chat, logs)   │
│    /health            → liveness                                   │
│                                                                    │
│  ┌───────────────────────── Service layer ─────────────────────┐  │
│  │ OntologyGenerator      GraphBuilderService                  │  │
│  │ ZepEntityReader        OasisProfileGenerator                │  │
│  │ SimulationManager      SimulationConfigGenerator            │  │
│  │ SimulationRunner ──────► OASIS subprocess (scripts/run_*.py)│  │
│  │ SimulationIPC (client/server, file-based command channel)  │  │
│  │ ReportAgent (ReAct) ──► ZepToolsService (graph toolset)     │  │
│  │ ZepGraphMemoryUpdater (dynamic temporal memory)             │  │
│  │ TextProcessor                                               │  │
│  └────────────────────────────────────────────────────────────┘  │
│  ┌─────────── Models ──────────┐  ┌──────────── Utils ─────────┐  │
│  │ Project   (JSON on disk)    │  │ LLMClient (OpenAI SDK)     │  │
│  │ Task      (in-memory single)│  │ FileParser (PDF/MD/TXT)    │  │
│  │ Simulation/Report state (fs)│  │ retry / locale / logger    │  │
│  └─────────────────────────────┘  │ zep_paging                 │  │
│                                    └────────────────────────────┘  │
└──────────────┬─────────────────────────────────┬──────────────────┘
               │ OpenAI SDK                       │ zep-cloud SDK
               ▼                                  ▼
        ┌────────────┐                      ┌─────────────┐
        │ LLM API    │                      │  Zep Cloud  │
        │ (OpenAI-   │                      │ graph +     │
        │  compatible)│                     │ memory      │
        └────────────┘                      └─────────────┘
```

### 2.2 Request lifecycle for long operations

The three expensive operations — **graph build**, **simulation prepare**, and **report generate** — follow the same asynchronous pattern:

```text
Client POST → handler validates → creates a Task (TaskManager) →
spawns daemon threading.Thread → returns {task_id} immediately (HTTP 200) →
client polls GET task/status → background thread updates Task progress →
on success: Task COMPLETED with result; on error: Task FAILED with message
```

> [!NOTE]
> Because progress lives in the in-memory `TaskManager` singleton, **restarting the backend loses all in-flight task state.** Persisted artifacts (the project, simulation files, report sections on disk) survive; the task *status objects* do not. The prepare/report endpoints mitigate this by also reconstructing "already done?" answers from on-disk files (`_check_simulation_prepared`, `get_report_by_simulation`).

### 2.3 The five-stage pipeline mapped to code

| Stage | Entry endpoint | Primary services | Output |
|-------|----------------|------------------|--------|
| 1. Graph Building | `POST /api/graph/ontology/generate`, `POST /api/graph/build` | `OntologyGenerator`, `GraphBuilderService`, `TextProcessor`, `FileParser` | Zep graph (`graph_id`) |
| 2. Environment Setup | `POST /api/simulation/prepare` | `ZepEntityReader`, `OasisProfileGenerator`, `SimulationConfigGenerator`, `SimulationManager` | `reddit_profiles.json`, `twitter_profiles.csv`, `simulation_config.json` |
| 3. Simulation | `POST /api/simulation/start` / `/stop` | `SimulationRunner` → OASIS subprocess, `SimulationIPC`, `ZepGraphMemoryUpdater` | `run_state.json`, action/round logs |
| 4. Report Generation | `POST /api/report/generate` | `ReportAgent`, `ReportManager`, `ZepToolsService` | report JSON + Markdown sections |
| 5. Deep Interaction | `POST /api/simulation/interview*`, `POST /api/report/chat` | `SimulationIPC` (live env), `ReportAgent` | interview answers, chat replies |

---

## 3. Requirements

### 3.1 Host requirements

| Component | Requirement |
|-----------|-------------|
| Node.js | ≥ 18.0.0 (frontend build + `concurrently` dev runner) |
| Python | ≥ 3.11 and < 3.13 (enforced by `pyproject.toml`) |
| uv | Latest (creates the backend virtualenv, resolves the lockfile) |
| OS | Cross-platform; the code has explicit Windows UTF-8 console handling (`run.py`, `logger.py`) |
| Network egress | Outbound HTTPS to the LLM endpoint and to Zep Cloud |

### 3.2 External service requirements

- **LLM endpoint** — any service speaking the OpenAI Chat Completions API. Must support `response_format={"type": "json_object"}` for the generation steps to be reliable.
- **Zep Cloud** — an account and API key. Used for graph storage, entity/edge extraction, and temporal memory.

### 3.3 Python dependency inventory

| Package | Pin | Role |
|---------|-----|------|
| `flask` | ≥ 3.0.0 | HTTP framework / app factory |
| `flask-cors` | ≥ 6.0.0 | CORS for `/api/*` |
| `openai` | ≥ 1.0.0 | LLM client (all providers via OpenAI format) |
| `zep-cloud` | == 3.13.0 | Knowledge graph + memory SDK |
| `camel-oasis` | == 0.2.5 | OASIS social-simulation engine |
| `camel-ai` | == 0.2.78 | CAMEL agent framework underlying OASIS |
| `PyMuPDF` | ≥ 1.24.0 | PDF text extraction (`fitz`) |
| `charset-normalizer`, `chardet` | ≥ 3.0 / ≥ 5.0 | Encoding detection for non-UTF-8 text |
| `python-dotenv` | ≥ 1.0.0 | `.env` loading |
| `pydantic` | ≥ 2.0.0 | Data validation |
| `pytest`, `pytest-asyncio` (dev) | ≥ 8.0 / ≥ 0.23 | Tests |

> [!WARNING]
> `zep-cloud`, `camel-oasis`, and `camel-ai` are **exact-pinned** (`==`). Do not float these — MiroFish calls version-specific APIs (e.g. `client.graph.node.get_by_graph_id` with `uuid_cursor` paging). Upgrading them is a deliberate, tested change, not a routine bump.

### 3.4 Frontend dependency inventory

| Package | Pin | Role |
|---------|-----|------|
| `vue` | ^3.5.24 | UI framework |
| `vue-router` | ^4.6.3 | SPA routing |
| `vue-i18n` | ^11.3.0 | Internationalization |
| `axios` | ^1.14.0 | HTTP client |
| `d3` | ^7.9.0 | Graph/data visualization |
| `vite` + `@vitejs/plugin-vue` (dev) | ^7.2.4 / ^6.0.1 | Build/dev tooling |

---

## 4. Configuration Mastery

All backend configuration is environment-variable driven and centralized in `backend/app/config.py` (class `Config`). The `.env` file is loaded from the **project root** (`MiroFish/.env`), resolved relative to `config.py`; if absent, environment variables are read directly (the production/Docker path).

### 4.1 Complete environment variable reference

| Variable | Default | Read in | Purpose |
|----------|---------|---------|---------|
| `LLM_API_KEY` | _(none — required)_ | `config.py:31` | LLM provider key. Startup validation fails without it. |
| `LLM_BASE_URL` | `https://api.openai.com/v1` | `config.py:32` | OpenAI-compatible endpoint base URL |
| `LLM_MODEL_NAME` | `gpt-4o-mini` | `config.py:33` | Model for ontology/persona/config generation and the ReportAgent |
| `ZEP_API_KEY` | _(none — required)_ | `config.py:36` | Zep Cloud key. Startup validation fails without it. |
| `SECRET_KEY` | `mirofish-secret-key` | `config.py:24` | Flask secret. **Override in any real deployment.** |
| `FLASK_DEBUG` | `True` | `config.py:25` | Debug mode (string `'true'` comparison). **Set to `False` outside development.** |
| `FLASK_HOST` | `0.0.0.0` | `run.py:40` | Bind address |
| `FLASK_PORT` | `5001` | `run.py:41` | Bind port |
| `OASIS_DEFAULT_MAX_ROUNDS` | `10` | `config.py:48` | Fallback round count when not otherwise set |
| `REPORT_AGENT_MAX_TOOL_CALLS` | `5` | `config.py:62` | Max tool calls the ReportAgent makes per section |
| `REPORT_AGENT_MAX_REFLECTION_ROUNDS` | `2` | `config.py:63` | ReportAgent self-reflection iterations |
| `REPORT_AGENT_TEMPERATURE` | `0.5` | `config.py:64` | Sampling temperature for report generation |
| `LLM_BOOST_API_KEY` / `LLM_BOOST_BASE_URL` / `LLM_BOOST_MODEL_NAME` | _(unset)_ | `.env.example` | Optional secondary "accelerated" model. Omit entirely if unused. |
| `WERKZEUG_RUN_MAIN` | _(set by Werkzeug)_ | `__init__.py:33` | Reloader-child detection to avoid double startup logging |

### 4.2 Non-env (code-level) configuration

These are set in `Config` and not env-overridable:

| Setting | Value | Meaning |
|---------|-------|---------|
| `MAX_CONTENT_LENGTH` | `50 * 1024 * 1024` | 50 MB max upload size |
| `UPLOAD_FOLDER` | `backend/uploads` | Root of all persisted state |
| `ALLOWED_EXTENSIONS` | `{pdf, md, txt, markdown}` | Accepted seed file types |
| `DEFAULT_CHUNK_SIZE` | `500` | Text chunk size for graph ingestion |
| `DEFAULT_CHUNK_OVERLAP` | `50` | Overlap between chunks |
| `OASIS_SIMULATION_DATA_DIR` | `backend/uploads/simulations` | Per-simulation working directories |
| `JSON_AS_ASCII` | `False` | Emit raw UTF-8 (so Chinese renders directly, not `\uXXXX`) |

### 4.3 OASIS platform action sets

The available agent actions per platform are defined in `Config` and injected into the simulation config:

| Platform | Actions |
|----------|---------|
| `OASIS_TWITTER_ACTIONS` | `CREATE_POST`, `LIKE_POST`, `REPOST`, `FOLLOW`, `DO_NOTHING`, `QUOTE_POST` |
| `OASIS_REDDIT_ACTIONS` | `LIKE_POST`, `DISLIKE_POST`, `CREATE_POST`, `CREATE_COMMENT`, `LIKE_COMMENT`, `DISLIKE_COMMENT`, `SEARCH_POSTS`, `SEARCH_USER`, `TREND`, `REFRESH`, `DO_NOTHING`, `FOLLOW`, `MUTE` |

### 4.4 Simulation configuration schema

`SimulationConfigGenerator` (an LLM call) produces a `SimulationParameters` object composed of:

- **`AgentActivityConfig`** — how active agents are.
- **`TimeSimulationConfig`** — `total_simulation_hours`, `minutes_per_round`; the recommended round count is derived as `total_hours × 60 / minutes_per_round`.
- **`EventConfig`** — scheduled events injected into the world.
- **`PlatformConfig`** — per-platform settings.

This serializes to `simulation_config.json` in the simulation directory.

> [!TIP]
> To force-regenerate personas or config after changing the model or requirement, pass `force_regenerate: true` to `/api/simulation/prepare`. Without it, MiroFish detects completed preparation via `_check_simulation_prepared()` and skips regeneration to save tokens.

---

## 5. Deployment

### 5.1 Source deployment

```bash
git clone https://github.com/666ghj/MiroFish.git
cd MiroFish
cp .env.example .env          # set LLM_API_KEY, ZEP_API_KEY
npm run setup:all             # root + frontend npm install, backend uv sync
npm run dev                   # concurrently runs backend (:5001) + frontend (:3000)
```

The root `package.json` scripts are the operational contract:

| Script | Command | Purpose |
|--------|---------|---------|
| `setup` | `npm install && cd frontend && npm install` | Node deps |
| `setup:backend` | `cd backend && uv sync` | Python deps |
| `setup:all` | `npm run setup && npm run setup:backend` | Everything |
| `dev` | `concurrently ... "npm run backend" "npm run frontend"` | Run both, kill-others |
| `backend` | `cd backend && uv run python run.py` | Backend only |
| `frontend` | `cd frontend && npm run dev` | Vite only |
| `build` | `cd frontend && npm run build` | Production frontend build |

### 5.2 Docker deployment

```yaml
# docker-compose.yml (as shipped)
services:
  mirofish:
    image: ghcr.io/666ghj/mirofish:latest
    container_name: mirofish
    env_file:
      - .env
    ports:
      - "3000:3000"
      - "5001:5001"
    restart: unless-stopped
    volumes:
      - ./backend/uploads:/app/backend/uploads
```

```bash
cp .env.example .env          # set your keys
docker compose up -d
```

The image (`Dockerfile`) is `python:3.11` with Node/npm added and `uv` copied from `ghcr.io/astral-sh/uv:0.9.26`. It installs Node deps (`npm ci`), frontend deps, and backend deps (`uv sync --frozen`), then runs `CMD ["npm", "run", "dev"]`.

> [!CAUTION]
> The container's command is `npm run dev` — the **Vite dev server and the Flask dev server**, not a production WSGI server or a built static bundle. This is convenient but is **not a hardened production setup**: no Gunicorn/uvicorn, `FLASK_DEBUG` may be on, and the dev server is single-process. For production, front it with a reverse proxy (TLS, auth), set `FLASK_DEBUG=False` and a strong `SECRET_KEY`, serve the built frontend (`npm run build`) as static files, and run Flask under a real WSGI server.

### 5.3 The frontend↔backend connection

- **Dev:** `vite.config.js` proxies `/api` → `http://localhost:5001` (with `changeOrigin`). The axios client also has `baseURL = import.meta.env.VITE_API_BASE_URL || 'http://localhost:5001'`.
- **Vite aliases:** `@` → `frontend/src`, `@locales` → repo-root `locales/`.
- To point the frontend at a non-local backend, set `VITE_API_BASE_URL` at build/dev time.

> [!NOTE]
> Locales are shared between frontend and backend: both read the repo-root `locales/` directory (`en.json`, `zh.json`, `languages.json`). The frontend imports them via the `@locales` alias and `import.meta.glob`; the backend loads them in `utils/locale.py`. Keep them in sync when adding a language.

---

## 6. API Reference

All endpoints are JSON over HTTP. Success responses follow `{"success": true, "data": ...}`; errors follow `{"success": false, "error": "...", "traceback": "..."}` with an appropriate status code (`400` validation, `404` not found, `500` internal). There is **no authentication**.

> [!NOTE]
> Paths below include the blueprint prefix. In the source, route decorators are blueprint-relative; the prefixes (`/api/graph`, `/api/simulation`, `/api/report`) are attached in `app/__init__.py` via `register_blueprint`.

### 6.1 Health

| Method | Path | Purpose |
|--------|------|---------|
| GET | `/health` | Liveness — `{"status": "ok", "service": "MiroFish Backend"}` |

### 6.2 Graph API (`/api/graph`)

| Method | Path | Purpose |
|--------|------|---------|
| POST | `/api/graph/ontology/generate` | **Step 1a.** `multipart/form-data`: `files` (PDF/MD/TXT), `simulation_requirement` (required), `project_name`, `additional_context`. Creates a project, extracts text, LLM-generates the ontology. Returns `project_id`, `ontology`, `analysis_summary`. |
| POST | `/api/graph/build` | **Step 1b.** JSON `{project_id, graph_name?, chunk_size?, chunk_overlap?, force?}`. Async; returns `task_id`. Builds the Zep graph from the project text. |
| GET | `/api/graph/project/<project_id>` | Get project details |
| GET | `/api/graph/project/list` | List projects (`?limit=`) |
| DELETE | `/api/graph/project/<project_id>` | Delete a project and its files |
| POST | `/api/graph/project/<project_id>/reset` | Reset to `ontology_generated`/`created` for a rebuild |
| GET | `/api/graph/task/<task_id>` | Poll an async task's status/progress |
| GET | `/api/graph/tasks` | List all tasks |
| GET | `/api/graph/data/<graph_id>` | Fetch graph nodes + edges |
| DELETE | `/api/graph/delete/<graph_id>` | Delete a Zep graph |

### 6.3 Simulation API (`/api/simulation`)

**Entity reading**

| Method | Path | Purpose |
|--------|------|---------|
| GET | `/api/simulation/entities/<graph_id>` | Filtered entities (`?entity_types=`, `?enrich=`) |
| GET | `/api/simulation/entities/<graph_id>/<entity_uuid>` | Single entity with context |
| GET | `/api/simulation/entities/<graph_id>/by-type/<entity_type>` | Entities of one type |

**Lifecycle**

| Method | Path | Purpose |
|--------|------|---------|
| POST | `/api/simulation/create` | JSON `{project_id, graph_id?, enable_twitter?, enable_reddit?}`. Creates a `sim_*`. |
| POST | `/api/simulation/prepare` | **Step 2.** Async. `{simulation_id, entity_types?, use_llm_for_profiles?, parallel_profile_count?, force_regenerate?}`. Reads entities → generates personas → writes config. |
| POST | `/api/simulation/prepare/status` | Poll prepare progress (`{task_id?, simulation_id?}`); also reports cached "already prepared" |
| GET | `/api/simulation/<simulation_id>` | Get simulation state (adds `run_instructions` when `ready`) |
| GET | `/api/simulation/list` | List simulations (`?project_id=`) |
| GET | `/api/simulation/history` | Enriched history for the home page (`?limit=`) |
| POST | `/api/simulation/generate-profiles` | (Re)generate agent profiles |

**Profiles & config inspection**

| Method | Path | Purpose |
|--------|------|---------|
| GET | `/api/simulation/<id>/profiles` | Profiles (`?platform=reddit\|twitter`) |
| GET | `/api/simulation/<id>/profiles/realtime` | Profiles while still generating (file-direct, with metadata) |
| GET | `/api/simulation/<id>/config` | Simulation config |
| GET | `/api/simulation/<id>/config/realtime` | Config while still generating |
| GET | `/api/simulation/<id>/config/download` | Download config JSON |
| GET | `/api/simulation/script/<script_name>/download` | Download a run script |

**Run control**

| Method | Path | Purpose |
|--------|------|---------|
| POST | `/api/simulation/start` | **Step 3.** `{simulation_id, platform?, max_rounds?, enable_graph_memory_update?}`. Launches the OASIS subprocess. |
| POST | `/api/simulation/stop` | Stop a running simulation |
| GET | `/api/simulation/<id>/run-status` | Current run status |
| GET | `/api/simulation/<id>/run-status/detail` | Run status + recent actions |
| GET | `/api/simulation/<id>/actions` | Action history (`?limit,offset,platform,agent_id,round_num`) |
| GET | `/api/simulation/<id>/timeline` | Per-round timeline (`?start_round,end_round`) |
| GET | `/api/simulation/<id>/agent-stats` | Per-agent statistics |
| GET | `/api/simulation/<id>/posts` | Generated posts (`?platform,limit,offset`) |
| GET | `/api/simulation/<id>/comments` | Generated comments |

**Interaction (Step 5)**

| Method | Path | Purpose |
|--------|------|---------|
| POST | `/api/simulation/interview` | Interview a single live agent |
| POST | `/api/simulation/interview/batch` | Interview a batch `{simulation_id, interviews:[{agent_id, prompt}]}` |
| POST | `/api/simulation/interview/all` | Interview every agent |
| POST | `/api/simulation/interview/history` | Retrieve interview history |
| POST | `/api/simulation/env-status` | Check whether the live env is alive `{simulation_id}` |
| POST | `/api/simulation/close-env` | Gracefully close the running env `{simulation_id, timeout?}` |

### 6.4 Report API (`/api/report`)

| Method | Path | Purpose |
|--------|------|---------|
| POST | `/api/report/generate` | **Step 4.** Async. `{simulation_id, force_regenerate?}`. Returns `report_id` + `task_id`. Reuses a completed report unless forced. |
| POST | `/api/report/generate/status` | Poll generation (`{task_id?, simulation_id?}`) |
| GET | `/api/report/<report_id>` | Full report |
| GET | `/api/report/by-simulation/<simulation_id>` | Report for a simulation |
| GET | `/api/report/list` | List reports (`?simulation_id,limit`) |
| GET | `/api/report/<report_id>/download` | Download Markdown |
| DELETE | `/api/report/<report_id>` | Delete a report |
| POST | `/api/report/chat` | Chat with the ReportAgent `{simulation_id, message, chat_history?}` |
| GET | `/api/report/<report_id>/progress` | Live generation progress |
| GET | `/api/report/<report_id>/sections` | Sections generated so far (stream-as-you-go) |
| GET | `/api/report/<report_id>/section/<int:index>` | One section |
| GET | `/api/report/check/<simulation_id>` | Has-report + `interview_unlocked` flag |
| GET | `/api/report/<report_id>/agent-log` | ReportAgent structured action log (`?from_line=`) |
| GET | `/api/report/<report_id>/agent-log/stream` | Full agent log |
| GET | `/api/report/<report_id>/console-log` | Console-style log (`?from_line=`) |
| GET | `/api/report/<report_id>/console-log/stream` | Full console log |
| POST | `/api/report/tools/search` | Debug: graph search `{graph_id, query, limit?}` |
| POST | `/api/report/tools/statistics` | Debug: graph statistics `{graph_id}` |

---

## 7. Operations

### 7.1 Starting and stopping

```bash
npm run dev          # both services (foreground, color-coded)
npm run backend      # backend only
npm run frontend     # frontend only
docker compose up -d # detached Docker
docker compose down  # stop Docker
```

The backend registers a cleanup hook (`SimulationRunner.register_cleanup()`) on startup so that **all OASIS subprocesses are terminated when the server shuts down.** A clean shutdown of the Flask process therefore reaps child simulations.

### 7.2 Data layout on disk

Everything persists under `backend/uploads/` (the Docker bind mount). Knowing this layout is essential for backup, debugging, and cleanup.

```text
backend/uploads/
├── projects/
│   └── <project_id>/
│       ├── project.json          # Project metadata + ontology
│       ├── extracted_text.txt    # Concatenated seed text
│       └── files/                # Uploaded seed files (renamed)
├── simulations/
│   └── <simulation_id>/
│       ├── state.json            # SimulationState
│       ├── simulation_config.json
│       ├── reddit_profiles.json
│       ├── twitter_profiles.csv
│       └── run_state.json        # Round/action progress
└── reports/
    └── <report_id>/
        ├── meta.json
        └── section_NN.md         # Streamed report sections
```

- **Project IDs:** `proj_<12 hex>`  ·  **Reports:** `report_<12 hex>`  ·  **Tasks:** UUID4  ·  **Simulations:** `sim_*`.

### 7.3 Backup and restore

Because all durable state is files, backup is a directory copy:

```bash
tar czf mirofish-backup-$(date +%F).tgz backend/uploads/
```

Restore by extracting back into `backend/uploads/`. There is no database to dump. Note that the **Zep graphs live in Zep Cloud**, not on disk — a local backup captures projects/simulations/reports but not the remote graph data.

### 7.4 Logging

`utils/logger.py` configures a logger named `mirofish` with two sinks:

- **File:** `backend/logs/YYYY-MM-DD.log`, `RotatingFileHandler`, 10 MB × 5 backups, `DEBUG` level, detailed format.
- **Console:** `INFO` and above, concise format, forced UTF-8 on Windows.

Every request is logged (`before_request`/`after_request`); request bodies are logged at `DEBUG` when JSON.

> [!WARNING]
> Request-body `DEBUG` logging writes inbound JSON to disk. With `FLASK_DEBUG=True` (the default) this can persist user-supplied content and, potentially, secrets passed in payloads. Lower the log level and disable debug for any shared deployment.

### 7.5 Housekeeping

- `TaskManager.cleanup_old_tasks(max_age_hours=24)` prunes completed/failed tasks older than a day from memory.
- `SimulationRunner` exposes `cleanup_simulation_logs` / `cleanup_all_simulations` for log hygiene.
- Old `projects/`, `simulations/`, and `reports/` directories are **not** auto-purged — manage disk usage yourself.

---

## 8. Internals — Module Reference

### 8.1 Services (`backend/app/services/`)

| Module / class | Responsibility |
|----------------|----------------|
| `ontology_generator.py` · `OntologyGenerator` | LLM-generates `entity_types` + `edge_types` from seed text + requirement; can emit Python code for the schema |
| `graph_builder.py` · `GraphBuilderService` | Creates the Zep graph, sets the ontology, batches text into episodes (`add_text_batches`, batch_size 3), waits for Zep processing, fetches graph data |
| `zep_entity_reader.py` · `ZepEntityReader` | Reads nodes/edges from a Zep graph and filters to defined entity types (`filter_defined_entities`, `get_entities_by_type`, `get_entity_with_context`) |
| `oasis_profile_generator.py` · `OasisProfileGenerator` | Turns entities into `OasisAgentProfile`s with `to_reddit_format` / `to_twitter_format`; supports parallel generation |
| `simulation_config_generator.py` · `SimulationConfigGenerator` | LLM-derives `SimulationParameters` (activity, time, events, platform configs) |
| `simulation_manager.py` · `SimulationManager` | Orchestrates create/prepare; owns `SimulationState` and its persistence; produces run instructions |
| `simulation_runner.py` · `SimulationRunner` | Launches/stops the OASIS subprocess, tracks `SimulationRunState`, rounds, `AgentAction`s; registers global cleanup |
| `simulation_ipc.py` · `SimulationIPCClient/Server` | File/command-channel IPC between Flask and the running OASIS process — used for live interviews and graceful env close (`CommandType`, `IPCCommand`, `IPCResponse`) |
| `report_agent.py` · `ReportAgent`, `ReportManager`, `Report*` | ReAct report generator with planning, sectioning, tool calls, reflection; `ReportManager` persists sections, progress, and dual logs (structured agent log + console log) |
| `zep_tools.py` · `ZepToolsService` | The graph toolset the ReportAgent calls: `search_graph`, node/edge readers, `get_entity_summary`, `get_graph_statistics`, plus richer results (`InsightForgeResult`, `PanoramaResult`, `AgentInterview`) |
| `zep_graph_memory_updater.py` · `ZepGraphMemoryUpdater`, `ZepGraphMemoryManager` | Streams simulation activity back into the graph as temporal episodes during a run |
| `text_processor.py` · `TextProcessor` | Preprocess, chunk (sentence-boundary aware), and stat text |

### 8.2 Models (`backend/app/models/`)

- **`Project`** (`project.py`) — a dataclass persisted as `project.json`. Tracks status (`created` → `ontology_generated` → `graph_building` → `graph_completed` / `failed`), files, ontology, `graph_id`, chunk settings. `ProjectManager` handles filesystem CRUD and file storage (uploaded files renamed to `<8 hex><ext>`).
- **`Task`** (`task.py`) — an in-memory async-task record (status, progress 0–100, message, result, error). `TaskManager` is a **thread-safe singleton** (`__new__` + `threading.Lock`). Not persisted — see the restart caveat in [§2.2](#22-request-lifecycle-for-long-operations).

### 8.3 Utilities (`backend/app/utils/`)

| Module | Notes |
|--------|-------|
| `file_parser.py` · `FileParser` | PDF via PyMuPDF; MD/TXT via `_read_text_with_fallback` (UTF-8 → charset-normalizer → chardet → UTF-8+replace). Also `split_text_into_chunks`. |
| `llm_client.py` · `LLMClient` | Wraps the OpenAI SDK. `chat()` strips `<think>...</think>`; `chat_json()` strips Markdown fences and parses JSON, raising `LLM返回的JSON格式无效` on failure. |
| `locale.py` | Thread-local i18n. `get_locale()` uses the `Accept-Language` header in request context, else thread-local (set via `set_locale()` at the top of background threads). `t(key, **kwargs)` resolves dotted keys with `zh` fallback. |
| `logger.py` | Rotating file + console logging; Windows UTF-8 handling. |
| `retry.py` | `retry_with_backoff` / `_async` decorators and `RetryableAPIClient` (exponential backoff + jitter, batch-with-per-item-retry). |
| `zep_paging.py` | UUID-cursor pagination for Zep node/edge listing, with per-page retry on transient errors. Caps nodes at 2000. |

### 8.4 Frontend structure (`frontend/src/`)

- **Entry:** `main.js` mounts `App.vue` with router + i18n. `App.vue` is just `<router-view/>`.
- **Routing (`router/index.js`):** `Home` (`/`), `Process` (`/process/:projectId`, = `MainView`), `Simulation` (`/simulation/:simulationId`), `SimulationRun` (`/simulation/:simulationId/start`), `Report` (`/report/:reportId`), `Interaction` (`/interaction/:reportId`).
- **API layer (`api/`):** `index.js` is the axios instance (5-min timeout, `Accept-Language` injection, response unwrapping, `requestWithRetry` with exponential backoff). `graph.js`, `simulation.js`, `report.js` are typed wrappers over the backend endpoints.
- **Components:** `Step1GraphBuild` … `Step5Interaction`, plus `GraphPanel` (D3), `HistoryDatabase`, `LanguageSwitcher`.

> [!NOTE]
> The axios client unwraps `response.data` and rejects when `res.success === false`, so frontend callers receive the `data` object directly and catch errors as thrown `Error`s. Any new endpoint must follow the `{success, data|error}` envelope to stay compatible.

---

## 9. Security Analysis

> [!CAUTION]
> MiroFish ships with **no authentication, no authorization, and permissive CORS.** The following are not theoretical — they are properties of the code as written. Deploy accordingly.

| Area | Finding | Recommendation |
|------|---------|----------------|
| **AuthN/AuthZ** | None. Every endpoint is open; any caller can create/read/delete any project, simulation, or report. | Put MiroFish behind an authenticating reverse proxy or add an auth middleware before exposing it. |
| **CORS** | `CORS(app, resources={r"/api/*": {"origins": "*"}})` — any origin may call the API. | Restrict `origins` to known frontends in production. |
| **Debug mode** | `FLASK_DEBUG` defaults to `True`; error responses include full `traceback`. | Set `FLASK_DEBUG=False`; strip tracebacks from client responses. |
| **Secret key** | Defaults to the literal `mirofish-secret-key`. | Always set a strong random `SECRET_KEY`. |
| **Secrets handling** | LLM/Zep keys are read from `.env`/env and used server-side (not exposed to the client). Request bodies are DEBUG-logged. | Keep `.env` out of version control (it is in `.dockerignore`/`.gitignore`); reduce body logging. |
| **File upload** | Extension-allowlisted (`pdf/md/txt/markdown`), 50 MB cap, uploaded files renamed to random names. PDF parsing via PyMuPDF. | Reasonable baseline; consider content-type verification and malware scanning for untrusted users. |
| **SSRF/egress** | `LLM_BASE_URL` is operator-controlled, so no user-driven SSRF; but the server makes outbound calls to whatever endpoint is configured. | Pin `LLM_BASE_URL` to trusted providers. |
| **Subprocess execution** | OASIS runs as a child process from `backend/scripts/`; script paths are validated (`脚本不存在` guard) and not user-supplied. | Keep `scripts/` read-only and unmodifiable by API callers. |
| **Server binding** | Default `FLASK_HOST=0.0.0.0` binds all interfaces. | Bind to `127.0.0.1` behind a proxy, or firewall port 5001. |

---

## 10. Code Quality & Conventions

**Strengths**

- **Clean layering:** API → services → models/utils with single-responsibility modules. The 60 endpoints group cleanly into three blueprints.
- **Consistent async pattern:** every long task returns a `task_id` and runs in a daemon thread with a uniform progress-callback shape.
- **Robustness:** centralized retry/backoff (`retry.py`, `RetryableAPIClient`), Zep paging with per-page retry, encoding fallbacks for text, and idempotent prepare/report (skip-if-done from on-disk state).
- **i18n done thread-aware:** locale captured before spawning background threads (`current_locale = get_locale(); set_locale(current_locale)`), a subtle correctness detail many codebases miss.
- **LLM defensiveness:** stripping `<think>` blocks and Markdown fences before JSON parsing acknowledges real model behavior.

**Risks & rough edges**

- **In-memory `TaskManager`** loses progress on restart; partially mitigated by on-disk reconstruction but not for arbitrary in-flight tasks.
- **No tests are wired into a pipeline** beyond the `pytest`/`pytest-asyncio` dev deps; coverage of the service layer is unverified from the source tree.
- **Large handler files:** `simulation.py` is ~2,600 lines; route handlers carry nontrivial business logic that could move into services.
- **Tracebacks leak to clients** by default, coupling debuggability with information disclosure.
- **Bilingual code:** error strings and many docstrings are Chinese; non-Chinese-reading maintainers should rely on the `locales/` keys.

---

## 11. Testing

The backend declares dev tooling (`pytest`, `pytest-asyncio`, `pipreqs`) under `[project.optional-dependencies].dev` and `[dependency-groups].dev`. There are test scripts under `backend/scripts/` (e.g. `test_profile_format.py`).

To run tests after installing dev deps:

```bash
cd backend
uv sync --extra dev
uv run pytest
```

> [!NOTE]
> Because the system depends on live external services (LLM, Zep) and on the OASIS subprocess, meaningful end-to-end testing requires either real credentials or mocks for the OpenAI and zep-cloud clients. Unit-test the pure logic (`TextProcessor` chunking, `FileParser`, `retry`, `locale`, the dataclass `to_dict`/`from_dict` round-trips) without external calls.

---

## 12. Troubleshooting (Operations)

### Backend exits immediately with `配置错误`

`Config.validate()` returns errors when `LLM_API_KEY` or `ZEP_API_KEY` is unset. The process prints the missing keys and `sys.exit(1)`. Fix `.env` (project root) and restart.

### Graph build task fails

Check `backend/logs/` for the `mirofish.build` logger. Common causes: missing/invalid `ZEP_API_KEY`, Zep rate limits, or LLM JSON failures during ontology use. The project status flips to `failed` with the error stored; reset and rebuild.

### `500` on ontology generation with reasoning models

Reasoning models (Qwen3, DeepSeek-R1) emit content that breaks strict JSON (community issue #642). `LLMClient` strips `<think>` and fences, but switch `LLM_MODEL_NAME` to a standard chat model for the generation steps.

### PDF upload → `Unexpected non-whitespace character after JSON`

Reported in issues #672/#670 (byte-identical error position across attempts). Workaround: upload Markdown/text instead of the offending PDF, or regenerate the PDF. Keep files under 50 MB.

### Simulation never reaches `ready`

Inspect the simulation directory under `backend/uploads/simulations/<id>/`. `_check_simulation_prepared` requires `state.json`, `simulation_config.json`, `reddit_profiles.json`, and `twitter_profiles.csv` to all exist, with `config_generated: true`. Missing files indicate the prepare task failed mid-way — re-run prepare (optionally `force_regenerate`).

### Interviews fail with "environment not running"

Live interviews go over IPC to the OASIS subprocess (`无法执行Interview` errors come from `simulation_runner.py`). If the env was closed or crashed, restart the simulation. Use `/api/simulation/env-status` to check liveness.

### Repeated `429`/`500` under load

Provider rate limiting (issue #662). Reduce rounds/agents, throttle, or upgrade your LLM tier. The built-in backoff helps but cannot exceed your provider's quota.

### Orphaned OASIS processes

A clean Flask shutdown reaps children via the registered cleanup hook. If the server was killed `-9`, check for stray Python processes from `backend/scripts/run_*_simulation.py` and terminate them; then use `cleanup_all_simulations` housekeeping.

---

## 13. Potential Improvements

These are derived from the code and from community issues; none are implemented as of `96096ea`.

- **Persist task state** (SQLite or a JSON journal) so progress survives restarts instead of living only in the `TaskManager` singleton.
- **Production runtime:** ship a built frontend served as static assets and run Flask under a WSGI server (Gunicorn/uvicorn-worker) rather than `npm run dev` in the container.
- **Add an auth layer** and tighten CORS before any multi-user or public exposure.
- **Stop leaking tracebacks** to clients when not in debug.
- **Report export formats** (issue #636 requests saving the final report) — a Markdown download exists; PDF/HTML export and a clearer in-UI "save report" affordance would close the gap.
- **OpenZep / self-hosted Zep support** (issue #183) to escape Zep Cloud quota limits.
- **Thin the route handlers** by moving business logic from `simulation.py`/`report.py` into the service layer for testability.
- **Resilience for reasoning models:** a stricter JSON-repair or function-calling path would reduce the #642 class of failures.

---

## 14. Quick Reference

### Ports & URLs

| What | Value |
|------|-------|
| Frontend | `http://localhost:3000` |
| Backend | `http://localhost:5001` |
| Health | `GET /health` |
| API roots | `/api/graph`, `/api/simulation`, `/api/report` |

### Key files

| File | Role |
|------|------|
| `.env` | Secrets + config (project root) |
| `backend/run.py` | Backend entry point |
| `backend/app/__init__.py` | Flask app factory, blueprint mounting, CORS |
| `backend/app/config.py` | All configuration |
| `frontend/vite.config.js` | Dev server port 3000, `/api` proxy, aliases |
| `docker-compose.yml` / `Dockerfile` | Container deployment |
| `backend/uploads/` | All persisted state |
| `locales/` | Shared i18n (en, zh) |

### ID formats

| Entity | Format |
|--------|--------|
| Project | `proj_<12 hex>` |
| Report | `report_<12 hex>` |
| Task | UUID4 |
| Simulation | `sim_*` |

### Operational commands

```bash
npm run dev                       # run both services
docker compose up -d              # run via Docker
curl localhost:5001/health        # liveness
tar czf backup.tgz backend/uploads/   # back up all state
```

---

_Generated June 2026 from 666ghj/MiroFish at 96096ea. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._