MiniCPM5-2B: Running OpenBMB's 2B On-Device Model Locally
MiniCPM5-2B packs 2.5B params, 131K context, and 4B-beating benchmarks. Here's how it works and how to run it via GGUF, MLX, or GPTQ.
What is MiniCPM5-2B?
MiniCPM5-2B is a 2.5-billion-parameter dense language model from OpenBMB, built specifically for local and on-device deployment rather than cloud serving. It’s the second release in the MiniCPM5 line (after MiniCPM5-1B), and according to OpenBMB’s own benchmarks it beats several larger 4B-class open models on coding, math, tool use, and long-context tasks, while running on far less hardware.
TL;DR
- MiniCPM5-2B has 2,516,756,480 total parameters (about 1.98 billion excluding embeddings), spread across 42 transformer layers using grouped-query attention (16 query heads, 2 key/value heads).
- The model supports a native context length of 131,072 tokens, which is unusual for a model this small and matters for agentic workflows that need to hold long tool logs or documents in context.
- OpenBMB reports an average benchmark score of 53.9 across code reasoning, math reasoning, tool use, and agentic tasks, ahead of Qwen3.5-4B (51.1), granite-4.2-3B (42.7), and LFM2.5-2.6B (33.2) in their comparison set.
- It ships in multiple quantized formats out of the box: GGUF for llama.cpp, Ollama, and LM Studio; MLX (4-bit) for Apple Silicon; and GPTQ (4-bit) for GPU-based quantized inference.
- OpenBMB released four training checkpoints (Base, Midtrain, SFT, and the final RL+OPD-tuned model), plus a DSpark draft model meant to accelerate inference through speculative decoding.
- The release is paired with open training datasets under the UltraData umbrella, covering web pretraining, code, agent SFT, and RL data, so the training recipe isn’t a black box.
- It’s licensed under Apache 2.0, architecturally a standard
LlamaForCausalLM, which means it slots into existing Llama-compatible tooling without custom loaders.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
How big is MiniCPM5-2B, really?
The “2B” in the name refers to roughly 2.5 billion total parameters, with about 1.98 billion of those outside the embedding layers. That’s small enough to run comfortably on a laptop GPU, a decent CPU, or a modern phone once quantized. The architecture is a 42-layer dense transformer using grouped-query attention with 16 query heads and only 2 key-value heads, a common trick for cutting KV-cache memory without gutting quality.
What stands out more than the parameter count is the context window: 131,072 tokens natively. Most models in the sub-3B range top out at 4K to 32K unless context is extended after the fact with tricks that degrade quality. MiniCPM5-2B builds long context in from the start, which matters if you’re feeding it long documents, multi-turn agent transcripts, or big tool-call histories on a device that can’t afford a second, larger model just for that job.
How does MiniCPM5-2B compare to 4B models?
OpenBMB’s benchmark comparison puts MiniCPM5-2B against both same-size peers (LFM2.5-2.6B, Qwen3.5-2B, Gemma-4-E2B-it) and larger reference models (Qwen3.5-4B, granite-4.2-3B, Nemotron-3-Nano-4B, Gemma-4-E4B-it, LFM2.5-8B-A1B). Within that set, MiniCPM5-2B posts the highest average score, 53.9, ahead of Qwen3.5-4B’s 51.1, despite having roughly half the parameters.
The gap is widest in code reasoning, math reasoning, long-context understanding, tool use, and several agentic benchmarks (general agent, search agent, coding agent). Instruction-following and general knowledge scores are closer to parity with the bigger models rather than clear wins. This pattern (strong on structured/agentic tasks, more middling on broad world knowledge) is typical of models trained with heavy emphasis on code and agent data rather than raw scale, and matches what OpenBMB describes in its own methodology: the training pipeline uses tiered code data management (UltraData-Code) and roughly 500,000 agent-specific SFT samples on top of general pretraining.
Benchmark comparisons published by a model’s own creator should be read with the usual caution: the comparison set and metrics are chosen by OpenBMB, not an independent third party. But the direction (a well-trained 2B beating some 4B models on specific task types) is consistent with a broader trend in 2025-era small models, where training data quality and task-specific post-training close a lot of the gap that used to require raw parameter count.
What formats does MiniCPM5-2B ship in?
OpenBMB released MiniCPM5-2B across a wide set of formats and checkpoints, which is more variety than most small-model releases offer:
- BF16 checkpoints: Base (pretraining only), Midtrain, SFT (post-supervised fine-tuning, pre-RL), and the final release model (post-trained with reinforcement learning plus something OpenBMB calls OPD).
- GGUF: for llama.cpp, Ollama, and LM Studio, the standard path for CPU and mixed CPU/GPU inference.
- MLX: a 4-bit build for Apple Silicon, aimed at Mac users running inference through Apple’s MLX framework.
- GPTQ: a 4-bit quantized version for GPU inference stacks that support GPTQ kernels.
- DSpark: a smaller draft model meant to pair with the full model for speculative decoding, which speeds up generation by having a tiny model draft tokens that the larger model verifies.
- LiteRT: a build compatible with Google’s LiteRT-LM runtime, aimed at mobile and edge deployment.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
Having the full checkpoint lineage (Base through final RL-tuned model) public is notable. It lets researchers or fine-tuners pick up the model before or after specific training stages, rather than only getting the final chat-tuned artifact.
Is MiniCPM5-2B worth running locally?
For anyone building local-first tools, coding assistants that run offline, or agent pipelines that need to stay under tight memory budgets, MiniCPM5-2B is a reasonable pick to test. The combination of a 131K context window, agent-oriented training data, and multiple ready-made quantized formats means there’s low friction to trying it: GGUF users can load it into Ollama or LM Studio without conversion, Mac users get a native MLX build, and GPU users have GPTQ.
The tradeoffs are the ones you’d expect from any 2B model: general world knowledge and open-ended conversation quality won’t match a 4B or larger model, even if task-specific benchmarks look competitive. The benchmark strengths are concentrated in code, math, tool use, and agentic tasks, which suggests it’s better suited to being a workhorse for structured tasks (function calling, code completion, retrieval-augmented answering) than a general chat assistant competing with much larger models on breadth.
How do the training data releases fit in?
OpenBMB didn’t just release weights. It published the datasets behind them under the UltraData name: UltraX (a web pretraining corpus), UltraData-Code (tiered L0-L3 code data), UltraData-SFT-Agent-2609 (500K agent training samples), and UltraData-RL-2609 (80,000+ reinforcement learning samples covering math, code, general knowledge, and long-context reasoning). This level of data transparency is uncommon even among open-weight releases, where “open” often stops at the weights themselves. For teams fine-tuning their own small models, these datasets are a usable reference for what “agent-capable” training data looks like at this scale.
Frequently Asked Questions
How many parameters does MiniCPM5-2B have?
It has 2,516,756,480 total parameters, with about 1.98 billion non-embedding parameters, across 42 transformer layers.
What context length does MiniCPM5-2B support?
It natively supports up to 131,072 tokens of context, which is large for a model in the 2B parameter class.
Can MiniCPM5-2B run on a Mac or phone?
Yes. OpenBMB provides an MLX build (4-bit) for Apple Silicon, a GGUF build for llama.cpp-based tools, and a LiteRT version aimed at mobile and edge runtimes.
How does MiniCPM5-2B compare to Qwen3.5-4B?
In OpenBMB’s own benchmark comparison, MiniCPM5-2B scored an average of 53.9 versus 51.1 for Qwen3.5-4B, with the largest gaps in code reasoning, math, tool use, and agentic tasks.
What license is MiniCPM5-2B released under?
It’s released under the Apache 2.0 license, which permits commercial use, modification, and redistribution.
