How to Run MiMo-V2.6-Pro-RL Locally (Hardware and Setup)
A deployment guide for Xiaomi's 1T-parameter MiMo-V2.6-Pro-RL model using SGLang or vLLM, covering tensor parallelism and hardware needs.
What is MiMo-V2.6-Pro-RL?
MiMo-V2.6-Pro-RL is Xiaomi’s flagship open-weight model in the MiMo-V2.6 series, a sparse Mixture-of-Experts (MoE) model with 1.02 trillion total parameters and 42 billion activated per token. Running it locally means serving a multi-node GPU cluster with tensor and expert parallelism through SGLang or vLLM, not loading a checkpoint on a single workstation. It supports text, image, video, and audio input with a 1M token context window, and it’s built for agentic, coding, and cybersecurity tasks rather than casual chat.
TL;DR
- MiMo-V2.6-Pro-RL is a 1.02T-parameter MoE model with 42B active parameters per forward pass, making it a large-scale deployment rather than a single-GPU project.
- Xiaomi trained it with a unified RL run across coding, general agents, vision, and cybersecurity, using large-batch asynchronous GRPO instead of separate domain-specific fine-tunes.
- SGLang is the officially recommended serving stack, with a documented two-node, 16-way tensor-parallel configuration using DeepEP for expert parallelism.
- vLLM is a supported alternative with a simpler single-node, 8-way tensor-parallel recipe and a purpose-built Docker image.
- Speculative decoding via a 5-layer EAGLE-style MTP module predicts multiple tokens per step to cut latency, and both serving stacks expose flags to enable it.
- A 1M token context window and native multimodal encoders (vision, audio) add substantial memory overhead beyond the base MoE weights.
- Benchmark scores on agent, coding, and cybersecurity suites are competitive with or close to Claude Opus 5 and GPT-5.6 class models on several tasks, though it trails on some cybersecurity and terminal benchmarks.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
What hardware do you need to run MiMo-V2.6-Pro-RL?
The official SGLang recipe targets a two-node cluster running 16-way tensor parallelism (--tp 16) with 2-way data parallelism (--dp 2) and 16-way expert parallelism (--ep 16). That implies a setup with 16 GPUs total, split across two nodes of 8 each, which is the standard topology for serving trillion-parameter MoE models with today’s high-memory accelerators (80GB-class GPUs like H100 or H200).
The vLLM path is lighter: an 8-way tensor-parallel configuration (--tensor-parallel-size 8) on a single node. This is more approachable for teams with one 8-GPU box, though it likely sacrifices some throughput and headroom compared to the two-node SGLang layout, especially at long context lengths.
Beyond raw parameter count, three things inflate memory and complexity:
- The 1M token context window, which multiplies KV cache size dramatically at scale and is why the SGLang config tunes
--chunked-prefill-size,--page-size, and--swa-full-tokens-ratiospecifically for long-context serving. - The multimodal encoders: a 681M-parameter vision transformer and a combined ~435M-parameter audio stack (AudioTokenizer plus patch encoder), both of which need their own memory and compute budget alongside the LLM backbone.
- The 5-layer speculative decoding module (MTP), which adds a small but non-trivial amount of extra compute per step in exchange for faster generation.
There’s no official minimum GPU count published as a hard requirement, but the documented recipes make clear this is a multi-GPU, likely multi-node deployment, not something that fits on consumer hardware or a single data-center GPU.
How do you serve MiMo-V2.6-Pro-RL with SGLang?
Xiaomi’s own documentation points to the SGLang cookbook as the preferred deployment path, using the lmsysorg/sglang:latest Docker image. The reference command for a two-node setup looks like this:
sglang serve \
--trust-remote-code \
--model-path XiaomiMiMo/MiMo-V2.6-Pro-RL \
--tp 16 \
--dp 2 \
--enable-dp-attention \
--mm-enable-dp-encoder \
--ep 16 \
--moe-a2a-backend deepep \
--moe-dense-tp-size 1 \
--mem-fraction-static 0.7 \
--max-running-requests 128 \
--chunked-prefill-size 32768 \
--page-size 64 \
--swa-full-tokens-ratio 0.3 \
--speculative-algorithm EAGLE \
--speculative-num-steps 3 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 4 \
--enable-multi-layer-eagle \
--reasoning-parser mimo \
--tool-call-parser mimo \
--host 0.0.0.0 \
--port 30000 \
--nnodes 2 \
--node-rank <node-rank> \
--dist-init-addr <node0-ip>:20000A few flags are worth understanding rather than copy-pasting blindly:
--moe-a2a-backend deepepselects DeepEP for all-to-all communication between experts, which matters at this scale because routing tokens efficiently across 384 total experts (8 activated per token) is a major bottleneck.--speculative-algorithm EAGLEwith--enable-multi-layer-eagleturns on the model’s built-in multi-token prediction drafter, which predicts several tokens ahead for parallel verification instead of generating strictly one token at a time.--swa-full-tokens-ratioand--chunked-prefill-sizeare tuning knobs for the hybrid sliding-window-attention (SWA) and global-attention (GA) backbone, which mixes local and full attention layers to make the 1M token context tractable.- The
mimoreasoning and tool-call parsers are required because the model emits structured reasoning traces and tool calls in a Xiaomi-specific format that generic parsers won’t handle correctly.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
Running this across two nodes requires setting --node-rank and --dist-init-addr correctly on each machine so they can find each other over the network, standard practice for any multi-node inference cluster.
How do you serve MiMo-V2.6-Pro-RL with vLLM?
vLLM offers a simpler, single-node path documented in its MiMo-V2.5 recipe (also applicable to V2.6), with a dedicated prebuilt image (vllm/vllm-openai:mimov25-cu129):
vllm serve XiaomiMiMo/MiMo-V2.6-Pro-RL \
--tensor-parallel-size 8 \
--trust-remote-code \
--gpu-memory-utilization 0.95 \
--max-model-len auto \
--reasoning-parser mimo \
--tool-call-parser mimo \
--enable-auto-tool-choice \
--generation-config vllmThis is meaningfully shorter than the SGLang command because it skips explicit expert-parallelism and speculative-decoding flags, relying on vLLM’s defaults and 8-way tensor parallelism on one node. The --gpu-memory-utilization 0.95 setting pushes memory usage aggressively, which is typical when serving a model this large on a fixed GPU count. Recommended sampling settings for both stacks are temperature=1.0 and top_p=0.95.
Is self-hosting MiMo-V2.6-Pro-RL worth it?
For most teams, probably not, unless there’s a specific reason to run it on-premises. The model is also available through Xiaomi’s own AI Studio, MiMo Code, Xiaomi MiMo Desktop, the Xiaomi MiMo Open Platform API, and OpenRouter, all of which sidestep the multi-node GPU cluster entirely.
Self-hosting makes sense if you need data residency guarantees, want to fine-tune serving parameters for a specific workload, or are running at high enough volume that the infrastructure cost undercuts API pricing. It also matters if you’re integrating the model into a pipeline that needs guaranteed low-latency access to the full 1M token context or the multimodal encoders in ways an API might rate-limit or restrict.
On raw capability, MiMo-V2.6-Pro-RL’s benchmark numbers are competitive with frontier closed models on several agentic and coding tasks: it scores 89.9 on Terminal Bench 2.1 versus 89.1 for Claude Opus 5, and 76.9 on Toolathlon-Verified versus 80.6 for Opus 5. It also leads notably on cybersecurity benchmarks like CyberGym (94.0) and MiMo Cyber Bench (80.2), though it falls behind on ExploitBench and SEC Bench Pro compared to some peers. These numbers suggest it’s a strong general-purpose agent model with particular strength in code and cyber tasks, but not a uniform leader across every benchmark category.
Frequently Asked Questions
How many GPUs does MiMo-V2.6-Pro-RL need to run?
The official SGLang recipe uses 16 GPUs across two nodes (8 per node) with 16-way tensor parallelism. The vLLM recipe uses 8 GPUs on a single node with 8-way tensor parallelism. Exact GPU generation and VRAM per card aren’t specified in the model card, but the topology assumes high-memory data-center accelerators.
What’s the difference between MiMo-V2.6-Pro-RL and MiMo-V2.6-Flash-RL?
Both are part of the same MiMo-V2.6 series and share the same architecture family, but Pro-RL is the larger flagship checkpoint. Benchmark tables show Flash-RL scoring close to, and occasionally above, Pro-RL on some tasks (like CyberGym), while generally trailing on coding and general-agent benchmarks, suggesting Flash-RL is a smaller, faster variant.
Does MiMo-V2.6-Pro-RL support speculative decoding?
Yes. It ships with a built-in 5-layer sliding-window-attention multi-token-prediction (MTP) module, styled after DFlash, that predicts 7 subsequent tokens per forward pass. Both SGLang (via EAGLE-style flags) and vLLM can use this drafter to speed up generation.
Can I run MiMo-V2.6-Pro-RL without a GPU cluster?
Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
Not practically. At 1.02 trillion total parameters with 42 billion activated per token, plus multimodal encoders and a 1M token context window, the documented deployment paths all assume multi-GPU, often multi-node, serving infrastructure. There’s no quantized or consumer-hardware path described in the official deployment documentation.
What alternatives exist to self-hosting?
Xiaomi offers the model through its AI Studio, MiMo Code, MiMo Desktop app, and its own API platform, and it’s also listed on OpenRouter. These options avoid the infrastructure burden of running the trillion-parameter model yourself.
