GPT Academic Administrator & Developer Guide

1. Architecture Overview

GPT Academic is a Gradio-based web application designed for academic and programming productivity through LLM integration. The system provides a multi-model chat interface with a plugin architecture supporting 40+ specialized functions for code analysis, paper review, LaTeX processing, and more.

High-Level Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        User Browser                              │
└──────────────────────────────┬──────────────────────────────────┘
                               │ HTTP/WebSocket
┌──────────────────────────────▼──────────────────────────────────┐
│                    FastAPI + Gradio Server                        │
│  ┌─────────────┐  ┌──────────────┐  ┌────────────────────────┐ │
│  │ main.py     │  │ FastAPI      │  │ Gradio 3.32.15         │ │
│  │ (entry)     │  │ Server       │  │ (custom fork)          │ │
│  └──────┬──────┘  └──────┬───────┘  └────────────┬───────────┘ │
│         │                │                        │              │
│  ┌──────▼──────────────────────────────────────────▼───────────┐ │
│  │              shared_utils/ (17 modules)                      │ │
│  │  config_loader · cookie_manager · fastapi_server             │ │
│  │  fastapi_stream_server · handle_upload · key_pattern_manager │ │
│  └──────────────────────────┬──────────────────────────────────┘ │
│                             │                                    │
│  ┌──────────────────────────▼──────────────────────────────────┐ │
│  │           crazy_functions/ (49 plugin files)                  │ │
│  │  agent_fns · rag_fns · pdf_fns · latex_fns · review_fns     │ │
│  └──────────────────────────┬──────────────────────────────────┘ │
│                             │                                    │
│  ┌──────────────────────────▼──────────────────────────────────┐ │
│  │           request_llms/ (LLM Bridge Layer)                   │ │
│  │  bridge_all.py → 30+ model bridges                          │ │
│  │  key_manager.py · local_llm_class.py                         │ │
│  └──────────────────────────┬──────────────────────────────────┘ │
└──────────────────────────────┼──────────────────────────────────┘
                               │
          ┌────────────────────┼────────────────────┐
          ▼                    ▼                    ▼
   ┌─────────────┐    ┌─────────────┐    ┌─────────────────┐
   │ OpenAI/Azure│    │ Claude/     │    │ Local Models    │
   │ GPT-4o      │    │ Gemini/etc  │    │ Ollama/vLLM     │
   └─────────────┘    └─────────────┘    └─────────────────┘

Entry Point Flow

main.py
  ├── config.py (defaults) → config_private.py (overrides) → env vars (highest priority)
  ├── shared_utils/config_loader.py → get_conf() resolves final values
  ├── themes/theme.py → CSS/JS generation, dark mode
  ├── core_functional.py → basic function buttons (polish, translate, grammar)
  ├── crazy_functional.py → plugin registry
  └── Gradio Blocks UI construction → FastAPI mount → server launch

Configuration Priority Chain

Environment Variables (highest)
       ↓
config_private.py (user overrides, gitignored)
       ↓
config.py (defaults, committed)

The get_conf() function in shared_utils/config_loader.py resolves this chain at runtime for every configuration key access.

Component Responsibilities

2. System Requirements

Hardware Requirements

Software Dependencies

Important

The project uses a custom fork of Gradio 3.32.15. Using standard Gradio from PyPI will cause ModuleNotFoundError or AttributeError: module 'gradio' has no attribute 'Floating' errors. Always use the bundled version.

Python Package Requirements

Core dependencies from the project:

tiktoken>=0.3.3
openai
anthropic>=0.18.1
zhipuai==2.0.1
dashscope
gradio==3.32.15  # custom fork
fastapi==0.110
transformers>=4.27.1,<4.42
edge-tts>=7.0.0
spacy==3.7.4
pymupdf
nltk
llamaindex==0.10.68

Warning

The transformers library is pinned to >=4.27.1,<4.42. Versions 4.42+ introduce breaking changes for local model loading (ChatGLM, InternLM, etc.).

3. Configuration Reference

All configuration lives in config.py with overrides in config_private.py (gitignored) or environment variables. The get_conf() function resolves the final value at runtime.

API Keys and Model Configuration

Azure OpenAI Configuration

Server Configuration

File and Path Configuration

Feature Flags

Token and Rate Limiting

Plugin-Specific Configuration

Note

The ALLOW_RESET_CONFIG setting defaults to False. When enabled, users can modify configuration via natural language commands in the chat interface — this is a significant security consideration in multi-user deployments.

Model-Specific Settings

Settings are often model-prefixed for per-model override:

# Example: per-model token limits
MAX_TOKEN_OUTPUT = 512                    # global default
# Override in config_private.py:
# GPT4_MAX_TOKEN_OUTPUT = 4096

Environment Variable Override

Any setting from config.py can be overridden via environment variable of the same name:

export API_KEY="sk-xxxxx,sk-yyyyy"
export LLM_MODEL="gpt-4o"
export WEB_PORT="8080"
export AUTHENTICATION="[['admin','password123']]"

Caution

The AUTHENTICATION setting via environment variable must be a valid Python literal string representation of a list of lists. Malformed values will cause startup failures.

4. Deployment & Scaling

Docker Deployment (Recommended)

The project includes Dockerfile, docker-compose.yml, and .dockerignore at the repository root.

# Build and run
docker-compose up -d

# Or build manually
docker build -t gpt-academic .
docker run -d -p 8080:8080 \
  -e API_KEY="sk-your-key" \
  -e LLM_MODEL="gpt-4o" \
  -v ./gpt_log:/app/gpt_log \
  -v ./private_upload:/app/private_upload \
  gpt-academic

Docker Compose Configuration

version: '3'
services:
  gpt-academic:
    build: .
    ports:
      - "8080:8080"
    environment:
      - API_KEY=${API_KEY}
      - LLM_MODEL=${LLM_MODEL}
    volumes:
      - ./config_private.py:/app/config_private.py
      - ./gpt_log:/app/gpt_log
      - ./private_upload:/app/private_upload
    restart: unless-stopped

Bare Metal Deployment

# Clone repository
git clone https://github.com/binary-husky/gpt_academic.git
cd gpt_academic

# Install dependencies
pip install -r requirements.txt

# Create config_private.py
cat > config_private.py << 'EOF'
API_KEY = "sk-your-key-here"
LLM_MODEL = "gpt-4o"
WEB_PORT = 8080
AUTHENTICATION = [["admin", "secure_password"]]
EOF

# Start server
python main.py

Reverse Proxy (Nginx)

server {
    listen 443 ssl;
    server_name gpt-academic.example.com;

    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_read_timeout 300s;
        proxy_send_timeout 300s;
    }
}

Important

WebSocket support (Upgrade and Connection headers) is mandatory for the streaming chat interface. Without WebSocket passthrough, responses will not stream in real-time.

SSL/TLS Direct Configuration

The application supports native SSL without a reverse proxy:

# config_private.py
SSL_KEYFILE = "/path/to/privkey.pem"
SSL_CERTFILE = "/path/to/fullchain.pem"

Warning

If SSL_KEYFILE is set, SSL_CERTFILE must also be set. Setting only one raises a ValueError at startup.

Scaling Considerations

The application uses DEFAULT_WORKER_NUM = 3 for parallel processing within plugin executions. This controls how many LLM calls can run concurrently for a single user's operation (e.g., batch file analysis).

GROBID Service Deployment (for PDF plugins)

# Run GROBID for academic PDF parsing
docker run -d --name grobid \
  -p 8070:8070 \
  lfoppiano/grobid:latest

# Point GPT Academic to local GROBID
# config_private.py:
GROBID_URL = "http://localhost:8070"

Milvus Deployment (for RAG plugins)

# docker-compose for Milvus
docker-compose -f docker-compose-milvus.yml up -d

# The RAG plugins (crazy_functions/rag_fns/) connect to Milvus
# for vector storage and retrieval

5. Operations

Starting and Stopping

# Start (foreground)
python main.py

# Start (background with logging)
nohup python main.py > gpt_log/server.log 2>&1 &

# Docker
docker-compose up -d
docker-compose down

Log Management

Logs are written to gpt_log/ by default (configurable via PATH_LOGGING):

File Upload Management

User uploads are stored in private_upload/ (configurable via PATH_PRIVATE_UPLOAD):

private_upload/
  └── {session_hash}/
      └── {timestamp}/
          └── uploaded_files...

Tip

Implement periodic cleanup of private_upload/ in production. The application does not automatically purge old upload directories. A cron job removing directories older than 24-48 hours is recommended.

API Key Rotation

The key_manager.py module implements automatic key rotation:

1. Multiple keys are provided comma-separated in API_KEY
2. Keys are distributed round-robin across requests
3. Failed keys (auth errors, rate limits) are temporarily blacklisted
4. Blacklisted keys are retried after a cooldown period

# config_private.py — multiple keys for rotation
API_KEY = "sk-key1,sk-key2,sk-key3"

Health Monitoring

The application does not expose a dedicated health endpoint. Monitor via:

# Check if Gradio is responsive
curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/

# Check WebSocket connectivity
curl -s -o /dev/null -w "%{http_code}" \
  -H "Upgrade: websocket" \
  -H "Connection: Upgrade" \
  http://localhost:8080/queue/join

Auto-Update Mechanism

check_proxy.py includes an auto-update check that compares the local version against the remote repository. This runs at startup by default. In air-gapped or production environments, this behavior should be noted — it makes outbound HTTPS calls to GitHub.

Troubleshooting

Error: ModuleNotFoundError: No module named 'pkg_resources'

Cause: Python 3.12+ removed pkg_resources from the standard library. Some dependencies still reference it.

Solution:

pip install setuptools

Prevention: Pin Python to 3.11 or ensure setuptools is in requirements.

Error: AttributeError: module 'gradio' has no attribute 'Floating'

Cause: Standard Gradio installed instead of the project's custom 3.32.15 fork.

Solution:

pip uninstall gradio
# Install the bundled version from the project's requirements
pip install -r requirements.txt

Prevention: Always install from the project's pinned requirements, never pip install gradio directly.

Error: AssertionError on API_KEY format

Cause: API key does not match expected format patterns. The key_pattern_manager.py validates key prefixes: sk- (OpenAI), fk (api2d), azure- (Azure), etc.

Solution: Verify the key format matches the model being used. Check shared_utils/key_pattern_manager.py for the full pattern list.

Prevention: Use the correct key format for your provider. Multiple keys must be comma-separated with no spaces.

Error: ConnectionAbortedError during generation

Cause: Token limit exceeded — the response exceeded the model's context window.

Solution: Reduce MAX_TOKEN_OUTPUT or break input into smaller fragments via TOKEN_LIMIT_PER_FRAGMENT.

Error: GROBID_OFFLINE_EXCEPTION

Cause: GROBID service is unreachable at the configured GROBID_URL.

Solution:

# Check GROBID availability
curl -s http://localhost:8070/api/isalive

# If using the default HuggingFace Space, it may be sleeping
# Deploy a local instance for reliability

Error: RuntimeError about missing API keys

Cause: Attempting to use a model bridge without configuring its API key.

Solution: Set the appropriate key in config_private.py or environment variables. Each model family requires its specific key (e.g., ANTHROPIC_API_KEY for Claude, GOOGLE_API_KEY for Gemini).

6. Performance

Token Counting

The application uses tiktoken for accurate token counting with OpenAI models. For non-OpenAI models, approximate token counting is applied.

Timeout Configuration

TIMEOUT_SECONDS = 25        # Per-request timeout
MAX_RETRY = 2               # Retry count on failure
DEFAULT_WORKER_NUM = 3      # Parallel workers per operation

Tip

For large document analysis (papers, codebases), increase TIMEOUT_SECONDS to 120+ and DEFAULT_WORKER_NUM to 5-8 depending on available API rate limits.

Fragment Processing

Large inputs are automatically split into fragments based on:

• TOKEN_LIMIT_PER_FRAGMENT — max tokens per chunk
• INPUT_LIMIT_PER_FRAGMENT — max input tokens per call

This enables processing of documents that exceed a single model's context window by splitting, processing in parallel (up to DEFAULT_WORKER_NUM), and reassembling results.

Local Model Performance

Local models (ChatGLM, LLaMA, etc.) use the local_llm_class.py base class which implements:

• Multiprocess architecture to avoid GIL contention
• Separate process for model inference
• IPC via pipes for streaming tokens back to the main process

Main Process (Gradio/FastAPI)
    │
    ├── Pipe (IPC)
    │
    └── Child Process (GPU inference)
        └── Model loaded in VRAM

Caching

The application does not implement response caching by default. Each request hits the LLM provider. For production deployments handling repeated queries, consider adding a caching layer at the reverse proxy level.

7. Security

Authentication

Authentication is configured via the AUTHENTICATION setting:

# config_private.py
AUTHENTICATION = [
    ["admin", "strong_password_here"],
    ["user1", "another_password"],
]

When AUTHENTICATION is an empty list [], the application runs without authentication (default).

The logout endpoint is available at /academic_logout.

File Upload Security

shared_utils/handle_upload.py implements multiple protections:

1. Path traversal prevention — Uploaded filenames are sanitized; ../ sequences are rejected
2. Symlink detection — Symbolic links in upload paths are detected and rejected
3. Directory confinement — All file operations are validated to stay within the user's upload directory (private_upload/{session_hash}/)
4. FriendlyException — Validation failures raise user-facing errors without exposing internal paths

Caution

The file serving endpoint (/file/{path:path}) serves files from the application directory. In production, ensure the application runs with minimal filesystem permissions and that sensitive files (config_private.py, .env) are not accessible via this endpoint.

API Key Security

• Keys in config_private.py are gitignored by default
• The key rotation system in key_manager.py blacklists compromised keys automatically
• ALLOW_RESET_CONFIG = False prevents users from extracting or modifying keys via the chat interface

Warning

Never commit API keys to the repository. GitHub issue #2270 documents a case where a HuggingFace API key was accidentally exposed. Always use config_private.py or environment variables.

Network Security

Configuration Security

The ALLOW_RESET_CONFIG flag (default False) controls whether users can modify runtime configuration via natural language commands in the chat. When enabled:

• Users can change API_KEY, LLM_MODEL, and other settings via chat
• Changes persist for the session only
• This is intended for single-user/development scenarios

For multi-user production deployments, this must remain False.

Session Isolation

Each user session gets a unique hash (managed by shared_utils/cookie_manager.py). File uploads, chat history, and plugin state are isolated per session via this hash.

8. Developer Internals

Plugin Architecture

Plugin Types

1. Legacy plugins — Standalone functions decorated with @CatchException:

from toolbox import CatchException

@CatchException
def my_plugin(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
    # Plugin logic here
    pass

2. New-generation plugins — Class-based using GptAcademicPluginTemplate:

from crazy_functions.plugin_template.plugin_class_template import GptAcademicPluginTemplate

class MyPlugin(GptAcademicPluginTemplate):
    def define_arg_selection_menu(self):
        # Return up to 8 argument definitions
        pass

    def execute(self, txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
        # Plugin logic
        pass

Plugin Registration

Plugins are registered in crazy_functional.py:

from crazy_functions.my_plugin import my_plugin

function_plugins = {
    "My Plugin Name": {
        "Group": "编程",           # Plugin group
        "Color": "stop",          # Button color
        "AsButton": True,         # Show as button vs dropdown
        "Function": HotReload(my_plugin),  # Hot-reload wrapper
    }
}

Hot Reload

The HotReload() wrapper enables plugin modification without server restart. When PLUGIN_HOT_RELOAD = True, the wrapper re-imports the plugin module on each invocation.

Plugin Groups

Plugin Access to System Resources

Plugins receive these parameters:

• txt — User input text
• llm_kwargs — Model configuration (model name, temperature, etc.)
• plugin_kwargs — Plugin-specific arguments (up to 8 from arg selection menu)
• chatbot — Chat history (mutable, for streaming updates)
• history — Full conversation history
• system_prompt — Current system prompt
• user_request — Request metadata (including file upload paths)

LLM Bridge System

request_llms/bridge_all.py implements a routing layer that dispatches to model-specific bridges:

# Model prefix → Bridge mapping (simplified)
model_info = {
    "gpt-3.5-turbo":    bridge_chatgpt,
    "gpt-4":            bridge_chatgpt,
    "gpt-4o":           bridge_chatgpt,
    "claude-3-":        bridge_claude,
    "gemini-":          bridge_google_gemini,
    "glm-":             bridge_zhipu,
    "qwen-":            bridge_qwen,
    "deepseek-":        bridge_deepseek,
    "moonshot-":        bridge_moonshot,
    "ollama-":          bridge_ollama,
    "one-api-":         bridge_one_api,
    "openrouter-":      bridge_openrouter,
    "vllm-":            bridge_vllm,
}

Each bridge implements:

• predict() — Synchronous completion
• predict_no_ui_long_connection() — Long-running without UI updates
• Streaming response handling
• Error handling and retry logic

Key Pattern Manager

shared_utils/key_pattern_manager.py validates API key formats and routes to the correct provider:

This enables a single API_KEY field to auto-detect the provider and route accordingly.

Local LLM Architecture

request_llms/local_llm_class.py provides the base class for running local models:

┌─────────────────────────────────────┐
│         Main Process (Gradio)        │
│  ┌────────────────────────────────┐ │
│  │     LocalLLMClass instance      │ │
│  │  - Spawns child process         │ │
│  │  - Manages pipe communication   │ │
│  │  - Handles token streaming      │ │
│  └──────────────┬─────────────────┘ │
└─────────────────┼───────────────────┘
                  │ Pipe (IPC)
┌─────────────────▼───────────────────┐
│        Child Process (GPU)           │
│  - Loads model into VRAM             │
│  - Runs inference                    │
│  - Streams tokens back via pipe      │
└─────────────────────────────────────┘

This architecture avoids GIL contention between the web server and model inference.

Theme System

themes/theme.py provides dynamic theming:

• CSS generation based on configuration
• Dark mode toggle support
• Custom JavaScript injection via themes/*.js
• Gradio component styling overrides

FastAPI Endpoints

Cookie/Session Management

shared_utils/cookie_manager.py manages per-user session state:

• Session hash generation and validation
• State persistence across page reloads
• Plugin state isolation between users
• Upload directory assignment per session

Void Terminal

crazy_functions/vt_fns/ implements the Void Terminal — a stateful plugin execution environment that maintains context across multiple plugin invocations within a session.

9. Integration Points

OpenAI-Compatible APIs

Any OpenAI-compatible API can be integrated via API_URL_REDIRECT:

# config_private.py
API_URL_REDIRECT = {
    "https://api.openai.com/v1/chat/completions": "https://your-proxy.example.com/v1/chat/completions"
}

This enables integration with:

• Local OpenAI-compatible servers (LiteLLM, LocalAI, text-generation-webui)
• Corporate API gateways
• Rate-limiting proxies

Ollama Integration

# config_private.py
LLM_MODEL = "ollama-llama3"
AVAIL_LLM_MODELS = ["ollama-llama3", "ollama-codellama", "ollama-mistral"]
OLLAMA_HOST = "http://localhost:11434"  # if not default

Models prefixed with ollama- are automatically routed to the Ollama bridge.

vLLM Integration

# config_private.py
LLM_MODEL = "vllm-your-model"
# Requires vLLM server running with OpenAI-compatible API

One-API / OpenRouter

# One-API (unified gateway)
LLM_MODEL = "one-api-gpt4"
API_KEY = "your-one-api-key"

# OpenRouter
LLM_MODEL = "openrouter-anthropic/claude-3-opus"
API_KEY = "your-openrouter-key"

RAG Pipeline (LlamaIndex + Milvus)

The crazy_functions/rag_fns/ module implements retrieval-augmented generation:

Documents → LlamaIndex (chunking + embedding) → Milvus (vector store)
                                                       ↓
User Query → Embedding → Milvus (similarity search) → Context → LLM → Response

Configuration for RAG requires:

• Running Milvus instance
• LlamaIndex 0.10.68
• Embedding model access (OpenAI or local)

GROBID Integration (Academic PDF)

# config_private.py
GROBID_URL = "http://localhost:8070"  # Local GROBID instance
# Or use the public HuggingFace Space (unreliable):
# GROBID_URL = "https://qingxu98-grobid.hf.space"

GROBID provides structured extraction of academic papers (title, authors, abstract, sections, references).

Doc2X Integration (PDF)

# config_private.py
DOC2X_API_KEY = "your-doc2x-key"

Alternative PDF parsing service with better handling of complex layouts, tables, and equations.

Text-to-Speech (Edge TTS)

# Requires edge-tts>=7.0.0
# The /vits endpoint forwards TTS requests
# No additional configuration needed beyond package installation

AutoGen Agent System

crazy_functions/agent_fns/ integrates Microsoft AutoGen for multi-agent conversations:

• Agent-to-agent communication
• Task decomposition
• Tool use within agent chains

Note

The AutoGen integration uses the older AutoGen API. Compatibility with AutoGen 0.4+ (the AG2 rewrite) has not been verified.

Webhook / External Notification

The application does not include built-in webhook support for completion notifications. For integration with external systems (Slack, Discord, etc.), implement a wrapper around the /predict_user_input endpoint or add custom logic in a plugin.

Multi-Language Support

multi_language.py provides i18n support. The primary interface is Chinese with English as secondary. Translations are managed via the multi-language module.