MiniMind User Guide

MiniMind is an open-source project that trains a complete, working large language model (LLM) from scratch — for as little as a few dollars of GPU rental and roughly two hours on a single consumer GPU. It is both a from-zero reproduction of the full LLM training pipeline and a hands-on tutorial for learning how language models actually work.

Unlike fine-tuning projects that only adjust an existing model, MiniMind builds everything itself in native PyTorch: the model architecture (Dense and Mixture-of-Experts), the tokenizer, and every training stage — Pretrain, Supervised Fine-Tuning (SFT), LoRA, RLHF (DPO), RLAIF (PPO / GRPO / CISPO), Tool Use, Agentic RL, adaptive thinking, and knowledge distillation. The smallest mainline model is about 64M parameters — roughly 1/2700th the size of GPT-3 — small enough that an ordinary personal GPU can train and reproduce it quickly.

Note

MiniMind is released under the Apache License 2.0 and is completely free. The "two hours" figure refers to one SFT epoch on a single NVIDIA RTX 3090; the "a few dollars" figure refers to the corresponding GPU rental cost.

This guide walks you through installing MiniMind, running inference with a ready-made model, training your own model end-to-end, serving it through an OpenAI-compatible API, deploying it to popular inference engines, and resolving common problems.

Table of Contents

1. Quick Start
2. Requirements
3. Installation
4. Running Inference With a Pretrained Model
5. Training Your Own Model
6. Datasets
7. Fine-Tuning and Alignment (Optional Stages)
8. Tool Calling and Adaptive Thinking
9. Serving an OpenAI-Compatible API
10. Deploying to Third-Party Inference Engines
11. Resuming and Visualizing Training
12. Command Reference
13. Troubleshooting
14. FAQ

1. Quick Start

The fastest path to a working chat model is to download a published checkpoint and talk to it. The fastest path to training one is the "mini" data combo plus two scripts.

Chat with a published model in under five minutes:

# 1. Clone the repository and install dependencies
git clone --depth 1 https://github.com/jingyaogong/minimind
cd minimind && pip install -r requirements.txt

# 2. Download a published model (Transformers format)
modelscope download --model gongjy/minimind-3 --local_dir ./minimind-3

# 3. Chat from the command line
python eval_llm.py --load_from ./minimind-3

Train your own chat model from zero (the "MiniMind Zero" recipe):

# After downloading pretrain_t2t_mini.jsonl and sft_t2t_mini.jsonl into ./dataset/
cd trainer
python train_pretrain.py      # produces out/pretrain_768.pth
python train_full_sft.py      # produces out/full_sft_768.pth
python ../eval_llm.py --weight full_sft   # chat with your model

Tip

On a single RTX 3090, the Zero recipe (pretrain_t2t_mini + sft_t2t_mini, one epoch each) takes about 2.3 hours total and yields a small but genuinely conversational model. Use this combo first — switch to the full datasets only when you want to reproduce the mainline model.

2. Requirements

MiniMind is deliberately lightweight, but training a model still benefits from a GPU.

The author's reference configuration, for comparison:

• CPU: Intel Core i9-10980XE
• RAM: 128 GB
• GPU: 8 × NVIDIA RTX 3090 (24GB)
• Ubuntu 20.04, CUDA 12.2, Python 3.10.16

Important

torch and torchvision are intentionally commented out in requirements.txt. You must install PyTorch yourself with the build that matches your CUDA version. Installing the wrong CUDA build is the most common cause of "CUDA not available" problems.

Confirm your GPU is visible before training:

import torch
print(torch.cuda.is_available())

True

If this prints False, you can still run on CPU or MPS, but training speed and compatibility will differ substantially.

3. Installation

Standard (Linux / macOS)

git clone --depth 1 https://github.com/jingyaogong/minimind
cd minimind
pip install -r requirements.txt

With a faster mirror

git clone --depth 1 https://github.com/jingyaogong/minimind
cd minimind
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple

After installing the listed requirements, install PyTorch separately for your platform and CUDA version (see the PyTorch install selector). For example:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121

The repository ships with everything else you need to run inference and training: the model code (model/), the tokenizer (model/tokenizer.json), the dataset loaders (dataset/), and the training scripts (trainer/). No build step or package install is required — MiniMind is run directly as scripts, not installed as a Python package.

Note

There is no setup.py or pip install minimind. You run the scripts in place. Most training scripts assume you cd trainer first, because their default paths (such as ../dataset/... and ../out/...) are written relative to the trainer/ directory.

4. Running Inference With a Pretrained Model

You do not have to train anything to try MiniMind. Published checkpoints are available in two formats:

• Transformers format — a standard Hugging Face model directory (config.json, model.safetensors, tokenizer files). Load it by pointing --load_from at the directory.
• Native PyTorch format — a single *.pth weight file in ./out/, loaded by the from-scratch MiniMindForCausalLM class. Select it with --weight.

4.1 Download a published model

# Option 1 — ModelScope
modelscope download --model gongjy/minimind-3 --local_dir ./minimind-3
# Option 2 — Hugging Face
git clone https://huggingface.co/jingyaogong/minimind-3

4.2 Command-line chat

# Transformers-format model
python eval_llm.py --load_from ./minimind-3

# Native PyTorch weights (expects the matching .pth in ./out/)
python eval_llm.py --load_from ./model --weight full_sft

When the script starts, it asks how you want to drive it:

[0] Automatic test
[1] Manual input

Choose 0 to run a built-in list of sample prompts, or 1 to type your own. Responses stream token by token, and decoding speed is printed after each answer:

💬: Why is the sky blue?
🧠: The sky appears blue mainly because, when sunlight enters the atmosphere,
shorter blue wavelengths scatter more easily off air molecules...

[Speed]: 48.30 tokens/s

Useful inference flags (full list in Command Reference):

4.3 Web chat interface (optional)

A Streamlit chat UI lives in scripts/web_demo.py:

# Requires python>=3.10 and: pip install streamlit
# First copy a Transformers-format model folder into ./scripts/
cp -r minimind-3 ./scripts/minimind-3
cd scripts && streamlit run web_demo.py

Warning

web_demo.py scans the ./scripts/ directory for a subfolder containing weight files. If you do not copy a Transformers-format model into ./scripts/ first, the script will error because it has nothing to load.

5. Training Your Own Model

MiniMind's core promise is training from zero. The two required stages are pretraining and supervised fine-tuning; everything after that is optional refinement.

Pretrain ──▶ Full SFT ──▶ (optional) LoRA / DPO / RLAIF / Agentic RL / Distillation
(required)   (required)

All training scripts live in trainer/ and should be run from that directory.

5.1 Step 1 — Download training data

Download the dataset files from ModelScope or Hugging Face and place them in ./dataset/. To reproduce the Zero model quickly you only need two files:

./dataset/
├── pretrain_t2t_mini.jsonl   (~1.2 GB)  ← required for fast reproduction
└── sft_t2t_mini.jsonl        (~1.6 GB)  ← required for fast reproduction

See Datasets for the full list and recommended combinations.

5.2 Step 2 — Pretraining (required)

Pretraining teaches the model basic language patterns and world knowledge by predicting the next token across large amounts of text. This is the foundation everything else builds on.

cd trainer
# Single GPU
python train_pretrain.py
# Or, explicitly via torchrun (set --nproc_per_node to your GPU count)
torchrun --nproc_per_node 1 train_pretrain.py

The result is out/pretrain_768.pth (the 768 is the model's hidden dimension; new saves overwrite the old file).

# Quick sanity check of the pretrained model
python ../eval_llm.py --weight pretrain

5.3 Step 3 — Supervised fine-tuning (required)

SFT adapts the model to multi-turn conversation templates and continues injecting knowledge, response style, and behavior. It teaches the model the user / assistant / system / tool role structure and strengthens instruction following.

cd trainer
python train_full_sft.py        # produces out/full_sft_768.pth
python ../eval_llm.py --weight full_sft

After this stage you have a working chat model. The default full_sft weights already include basic Tool Calling ability, because tool-call samples are mixed into the mainline SFT data.

Tip

To train a Mixture-of-Experts (MoE) model instead of the Dense model, pass --use_moe 1 to every training stage and to eval_llm.py. MoE checkpoints are saved with a _moe suffix (for example, full_sft_768_moe.pth).

5.4 Multi-GPU training

Every script supports single-machine multi-GPU training with PyTorch DistributedDataParallel:

# Launch N-GPU training (extendable to multi-node)
torchrun --nproc_per_node N train_pretrain.py

6. Datasets

All datasets use the .jsonl format and drop into ./dataset/. Files marked ✨ are the recommended minimum for fast reproduction.

Data formats:

Pretraining data is plain text for next-token prediction:

{"text": "Transformers model context with self-attention, a key building block of modern LLMs."}

SFT and tool-use data are multi-turn conversations. Tool calls follow the OpenAI message format, with tools attached to the system message and tool_calls attached to the assistant message:

{
  "conversations": [
    {"role": "user", "content": "Translate 'hello world' to English"},
    {"role": "assistant", "content": "Hello World"}
  ]
}

DPO preference data pairs a preferred (chosen) and dispreferred (rejected) response:

{
  "chosen":   [{"role": "user", "content": "Q"}, {"role": "assistant", "content": "good answer"}],
  "rejected": [{"role": "user", "content": "Q"}, {"role": "assistant", "content": "bad answer"}]
}

Note

The training argument max_seq_len is measured in tokens, not characters. With MiniMind's tokenizer, Chinese text is roughly 1.5–1.7 characters per token and English is roughly 4–5 characters per token.

7. Fine-Tuning and Alignment (Optional Stages)

These stages refine an already-SFT'd model. All of them default to loading the full_sft weights (--from_weight full_sft) and run from the trainer/ directory.

7.1 LoRA — efficient domain adaptation

LoRA (Low-Rank Adaptation) adapts the model to a specialized domain by training only a small set of new low-rank parameters while keeping the base weights frozen — cheap, fast, and ideal for vertical use cases such as a medical or self-identity persona.

# Often runs comfortably even on CPU
python train_lora.py --lora_name lora_medical --data_path ../dataset/lora_medical.jsonl

Prepare your data as standard multi-turn conversations in a lora_xxx.jsonl file, then test the base model plus the LoRA weights together:

python ../eval_llm.py --weight full_sft --lora_weight lora_medical

LoRA weights are saved separately (for example lora_medical_768.pth) and can be merged back into a full model with scripts/convert_model.py:

cd scripts && python convert_model.py

Tip

The default LoRA rank is 16, and LoRA is applied only to square linear layers (those where input and output dimensions match). This keeps the added parameter count tiny relative to the base model.

7.2 DPO — preference alignment (RLHF)

Direct Preference Optimization aligns the model with human preferences using static preference pairs. It needs only the policy and a frozen reference model, so memory use is low and training is stable.

python train_dpo.py        # produces out/dpo_768.pth

Caution

The default DPO learning rate is intentionally tiny (4e-8). A learning rate above ~5e-8 tends to cause the model to forget its prior abilities. Increase it only with care.

7.3 RLAIF — PPO, GRPO, and CISPO

RLAIF (Reinforcement Learning from AI Feedback) continues training with automatically obtained reward signals rather than human labels. MiniMind ships from-scratch implementations of PPO, GRPO, and CISPO.

Reward model setup (required for these stages): download the InternLM2-1.8B-Reward model and place it as a sibling directory of the minimind/ project:

root/
├── minimind/                 # this project
└── internlm2-1_8b-reward/    # reward model

Train with GRPO (single-network, stable):

python train_grpo.py        # produces out/grpo_768.pth

Train with PPO (Actor + Critic):

python train_ppo.py         # produces out/ppo_actor_768.pth

Use CISPO: CISPO is implemented as a loss variant of GRPO. Set --loss_type cispo on train_grpo.py (this is in fact the default) rather than running a separate script.

Warning

On GPUs with 8GB of VRAM or less, train_grpo.py can run out of memory (reported in issue #758). Reduce --batch_size, --num_generations, and --max_gen_len to fit smaller cards. RLAIF on a ~0.1B model is also prone to "reward sparsity" — if rewards stay near zero, the learning signal vanishes. MiniMind mitigates this by using a continuous reward model rather than a binary right/wrong reward.

7.4 Knowledge distillation

train_distillation.py demonstrates white-box distillation: a student model continues training from SFT weights while matching a teacher model's output distribution via a combined cross-entropy + KL loss with temperature scaling.

python train_distillation.py

By default the teacher is a MoE model and the student is Dense; control the mix with --alpha (CE weight) and --temperature.

8. Tool Calling and Adaptive Thinking

MiniMind supports OpenAI-style tool calling and an adaptive "thinking" mode, both driven by the chat template — no separate model is required.

8.1 Tool calling

Because tool-call samples are mixed into the mainline SFT data, the default full_sft weights already perform basic tool calls. Test it directly:

python eval_toolcall.py --weight full_sft

💬: What time is it now?
🧠: <tool_call>{"name": "get_current_time", "arguments": {"timezone": "Asia/Shanghai"}}</tool_call>
📞 [Tool Calling]: get_current_time
✅ [Tool Called]: {"datetime": "2026-03-15 17:18:22", "timezone": "Asia/Shanghai"}
🧠: It is currently 17:18:22 on March 15, 2026.

The chat template expands tool calls and results into <tool_call>...</tool_call> and <tool_response>...</tool_response> segments automatically.

8.2 Adaptive thinking

Explicit chain-of-thought is controlled by a single open_thinking switch, available from the CLI, the OpenAI-compatible API, and the WebUI:

• open_thinking=0 (default): the template injects an empty <think>\n\n</think>, and the model tends to answer directly.
• open_thinking=1: the template injects a <think> opening tag, and the model produces explicit reasoning before its final answer.

python eval_llm.py --load_from ./minimind-3 --open_thinking 1

Through the OpenAI SDK:

response = client.chat.completions.create(
    model="minimind",
    messages=[{"role": "user", "content": "Who are you?"}],
    extra_body={"chat_template_kwargs": {"open_thinking": True}}
)

Note

When tool calling and explicit thinking are enabled at the same time, the small model does not reliably emit reasoning, because the training data lacks combined "reasoning + tool call" samples.

8.3 Agentic RL (advanced)

train_agent.py trains multi-turn Tool-Use behavior with GRPO/CISPO, executing tools between turns and rewarding the full trajectory. It can use either the native PyTorch rollout engine or an external SGLang server for higher-throughput sampling:

# Native PyTorch rollout
python train_agent.py        # produces out/agent_768.pth

# SGLang rollout (start the server first)
python -m sglang.launch_server --model-path ./minimind-3 --attention-backend triton --host 0.0.0.0 --port 8998
python train_agent.py --rollout_engine sglang --sglang_base_url http://localhost:8998 \
    --data_path ../dataset/agent_rl_math.jsonl --use_wandb

In the author's reported comparison on a light math tool-use benchmark, the agent weights solved 17/20 (85%) versus 12/20 (60%) for full_sft — though agent training can reduce factual stability on open-ended questions.

9. Serving an OpenAI-Compatible API

scripts/serve_openai_api.py exposes your model through an OpenAI-compatible chat endpoint, so you can plug MiniMind into tools like FastGPT, Open-WebUI, or Dify. It additionally supports the reasoning_content, tool_calls, and open_thinking fields.

Start the server:

cd scripts && python serve_openai_api.py

The server listens on http://0.0.0.0:8998. By default it loads native ../model torch weights with the full_sft prefix; point --load_from at a Transformers-format directory to serve that instead.

Test it with the bundled client:

cd scripts && python chat_api.py

Or call it directly with curl:

curl http://localhost:8998/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "model-identifier",
    "messages": [{ "role": "user", "content": "What is the tallest mountain in the world?" }],
    "temperature": 0.7,
    "max_tokens": 1024,
    "stream": true,
    "open_thinking": true
  }'

The endpoint supports both streaming ("stream": true, server-sent events) and non-streaming responses. When the model emits <think>...</think>, the server splits it into a separate reasoning_content field; when it emits <tool_call>, the server returns structured tool_calls.

10. Deploying to Third-Party Inference Engines

Because MiniMind's architecture is aligned with the Qwen3 ecosystem, published Transformers-format models work with common inference engines. All of the following expect a Transformers-format model directory.

ollama

# Use the author's published model directly
ollama run jingyaogong/minimind-3

Or load your own GGUF model by creating a minimind.modelfile, then:

ollama create -f minimind.modelfile minimind-local
ollama run minimind-local

vLLM

vllm serve /path/to/model --model-impl transformers --served-model-name "minimind" --port 8998

SGLang

python -m sglang.launch_server --model-path /path/to/model --attention-backend triton --host 0.0.0.0 --port 8998

llama.cpp

# Convert HF model to GGUF (see note below about tokenizer support)
python convert_hf_to_gguf.py /path/to/minimind-model
# Optional quantization
./build/bin/llama-quantize /path/to/model/xxxx.gguf /path/to/model/xxxx.q8.gguf Q8_0
# Run
./build/bin/llama-cli -m /path/to/model/xxxx.gguf

Important

For llama.cpp GGUF conversion, MiniMind's tokenizer is not recognized out of the box. Append a fallback in the get_vocab_base_pre function of convert_hf_to_gguf.py so the converter reuses a compatible pre-tokenizer (for example, qwen2).

Tip

vLLM, SGLang, and similar engines require a CUDA environment. Use llama.cpp or ollama for CPU-friendly local deployment. MNN is also supported for on-device (mobile) inference via llmexport.py.

If you trained native PyTorch weights, convert them to Transformers format first with scripts/convert_model.py before using these engines.

11. Resuming and Visualizing Training

11.1 Checkpoint resume

Every training script supports resumable checkpoints. Add --from_resume 1 to automatically detect and continue from the last saved checkpoint:

python train_pretrain.py --from_resume 1
python train_full_sft.py --from_resume 1

• Full checkpoints (model, optimizer, scheduler, and progress) are saved to ./checkpoints/ with the naming pattern <weight>_<dim>_resume.pth.
• Resume works across a changed number of GPUs — the step count is adjusted automatically.
• Weights-and-Biases / SwanLab run continuity is preserved.

Tip

Checkpoint resume is ideal for long or unstable training sessions — you never lose progress if training is interrupted.

11.2 Experiment tracking

Add --use_wandb to log training curves:

python train_pretrain.py --use_wandb

Note

MiniMind defaults to SwanLab for visualization (it is WandB-API-compatible and accessible where WandB may not be). In the code, this is done by aliasing the import: import swanlab as wandb.

12. Command Reference

Defaults below are the script defaults; override any of them on the command line. Common flags shared by all training scripts are listed once, followed by the parameters specific to each.

12.1 Flags common to all training scripts

12.2 Per-script highlights

12.3 Inference scripts

13. Troubleshooting

Error: CUDA out of memory during train_grpo.py

Cause: RLAIF training holds the policy, reference, and reward models plus multiple generated samples per prompt; on GPUs with ≤8GB VRAM this overflows memory (issue #758).

Solution: Lower --batch_size, --num_generations, and --max_gen_len. For example: python train_grpo.py --batch_size 1 --num_generations 4 --max_gen_len 512.

Prevention: Start RLAIF with the smallest viable settings and increase only if memory allows; prefer the Dense model over MoE for RL on small cards.

Error: AutoTokenizer.from_pretrained fails with a relative path

Cause: Newer versions of transformers / huggingface_hub reject certain relative paths when loading a tokenizer (issue #781).

Solution: Pass an absolute path to --load_from / --tokenizer_path, or run the script from a directory where the relative path resolves correctly.

Prevention: Prefer absolute paths for model and tokenizer directories.

Error: pyarrow / torch DLL conflict on Windows

Cause: On Windows, importing torch before datasets can trigger a pyarrow/torch DLL load conflict (issue #771).

Solution: This is already handled in the training scripts — they import datasets before other heavy imports. If you write your own script, do the same: import datasets first.

Prevention: Keep the existing import order in the trainer scripts.

Error: torch.cuda.is_available() returns False

Cause: PyTorch was installed without CUDA, or the CUDA build does not match your driver. Recall that torch is intentionally not pinned in requirements.txt.

Solution: Reinstall PyTorch with the correct CUDA build, e.g. pip install torch --index-url https://download.pytorch.org/whl/cu121.

Prevention: Always install the PyTorch build that matches your installed CUDA version.

Problem: web_demo.py errors on startup

Cause: The script scans ./scripts/ for a subfolder containing weight files and finds none.

Solution: Copy a Transformers-format model directory into ./scripts/ (e.g. cp -r minimind-3 ./scripts/) before running streamlit run web_demo.py.

Prevention: Always stage a model under ./scripts/ before launching the WebUI.

Problem: training data not found / wrong path

Cause: Training scripts use defaults relative to the trainer/ directory (such as ../dataset/...).

Solution: cd trainer before running, and place datasets in ./dataset/ at the repository root, or pass an explicit --data_path.

Prevention: Run training scripts from inside trainer/ as documented.

Problem: the model gives confidently wrong answers (hallucinations)

Cause: MiniMind is a ~0.1B-parameter model. Its factual knowledge and generalization are limited by design; this is expected for a model of this size.

Solution: Use the model for learning, experimentation, and tool-use tasks rather than as a factual knowledge base. RL-tuned weights (agent, grpo) may improve tool tasks while reducing open-ended factual stability.

Prevention: Match expectations to a tiny model — MiniMind exists to demonstrate the training pipeline, not to compete with billion-parameter models.

Problem: multi-GPU train_agent.py hangs at the end of training

Cause: A reported synchronization issue when distributed agent training exits (issue #790).

Solution: Check that issue thread for the current status and any fix; as a workaround, run agent training on a single GPU if you hit the hang.

Prevention: Track the project's issues for distributed-training fixes before relying on multi-GPU agent runs.

14. FAQ

Q: Do I need a powerful GPU to use MiniMind? A: To use a published model, no — CLI inference and the API server run on CPU or a modest GPU. To train from scratch comfortably, a single NVIDIA GPU (the author uses a 3090) is recommended. The Zero recipe trains in about two hours on one 3090.

Q: What is the difference between the pretrain, full_sft, dpo, grpo, ppo_actor, and agent weights? A: They are the checkpoints from successive training stages. pretrain knows language but cannot chat; full_sft is a usable chat model; dpo is preference-aligned; grpo/ppo_actor are RLAIF-tuned; agent is tuned for multi-turn tool use. Select one with --weight.

Q: Which datasets do I actually need? A: For a working chat model, only pretrain_t2t_mini.jsonl and sft_t2t_mini.jsonl. Add rlaif.jsonl, dpo.jsonl, or the agent datasets only for the corresponding optional stages.

Q: Dense or MoE — which should I train? A: Start with Dense (the default). MoE gives more capacity per active parameter but, in MiniMind's native PyTorch implementation, runs roughly 50% slower at the default 4 experts / top-1 setting. Add --use_moe 1 to every stage if you want MoE.

Q: Should I retrain the tokenizer? A: No. MiniMind ships a 6400-token tokenizer, and train_tokenizer.py is for learning only. Changing the tokenizer breaks compatibility with published weights, data formats, and the wider ecosystem.

Q: How do I make the model think before answering? A: Enable adaptive thinking with --open_thinking 1 (CLI) or chat_template_kwargs={"open_thinking": True} (API). The model then emits an explicit <think>...</think> block before its answer.

Q: How do I give the model a custom persona or domain knowledge? A: Train a LoRA adapter on conversation-format data (e.g. a self-identity or medical dataset), then load it with --lora_weight. This keeps the base model's general ability while adding your domain.

Q: Can I plug MiniMind into a chat UI like Open-WebUI? A: Yes. Run serve_openai_api.py to expose an OpenAI-compatible endpoint on port 8998, then point the UI at http://localhost:8998/v1.

Q: How do I run on multiple GPUs? A: Launch any training script with torchrun --nproc_per_node N <script>.py, where N is your GPU count. The scripts use DistributedDataParallel and can extend to multiple nodes.

Q: Can I extend the context length without retraining? A: Yes. Enable YaRN RoPE extrapolation at inference time with --inference_rope_scaling (native torch), or add a rope_scaling block to a Transformers model's config.json.

Q: Where do trained weights go? A: Into ./out/ as <prefix>_<hidden_size>.pth (with a _moe suffix for MoE models). Resume checkpoints go to ./checkpoints/.