How to Run MiMo-V2.6-Flash-RL Locally with vLLM or SGLang
A deployment guide to MiMo-V2.6-Flash-RL, Xiaomi's 309B MoE model with 15B active params, covering SGLang and vLLM setup.
What is MiMo-V2.6-Flash-RL?
MiMo-V2.6-Flash-RL is the efficiency-tuned checkpoint in Xiaomi’s MiMo-V2.6 model family, a sparse Mixture-of-Experts (MoE) model with 309 billion total parameters but only 15 billion active per forward pass. It handles text, image, video, and audio natively, supports a 1 million token context window, and ships with FP8 weights for lower-memory serving. It’s designed to run on SGLang or vLLM across a multi-GPU node, and it’s meant as the faster, cheaper sibling to the larger MiMo-V2.6-Pro-RL checkpoint.
TL;DR
- MiMo-V2.6-Flash-RL is a sparse MoE model with 309B total parameters and 15B activated per token, making it far cheaper to run than a dense model of similar total size.
- The model is natively omnimodal, accepting text, image, video, and audio in a single checkpoint, and supports context windows up to 1 million tokens.
- Xiaomi provides ready-made serving commands for both SGLang and vLLM, with SGLang recommended as the primary path via a published cookbook.
- The architecture uses a hybrid attention scheme (sliding window plus global attention layers) and a 5-layer speculative decoder (DFlash-style, EAGLE algorithm) to speed up inference.
- Benchmark results place Flash-RL slightly behind the Pro-RL variant and generally behind flagship closed models like Claude Opus 5 and GPT-5.6 Sol, but it’s competitive in cybersecurity tasks like CyberGym (95.1) and holds its own on agent benchmarks such as AutomationBench (52.3).
- Multi-GPU tensor and data parallelism are required out of the box: the reference SGLang command uses 8-way tensor parallelism with 2-way data parallelism, while the vLLM command uses 4-way tensor parallelism.
- The model is also reachable without self-hosting through Xiaomi’s AI Studio, MiMo Code, MiMo Desktop, the Xiaomi MiMo API platform, and OpenRouter.
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
What makes the Flash-RL checkpoint different from Pro-RL?
MiMo-V2.6 ships as two checkpoints: Pro-RL and Flash-RL. Both share the same underlying training recipe (a large-scale reinforcement learning pipeline described in Xiaomi’s technical report), but Flash-RL is the lighter-weight, faster-inference variant, positioned for deployments where latency and cost matter more than squeezing out the last few benchmark points. Across the published evaluation table, Flash-RL trails Pro-RL by small margins on most benchmarks (for example 67.9 vs 71.9 on DeepSWE v1.1, 61.2 vs 63.2 on MiMo Code Bench) but actually edges ahead on CyberGym (95.1 vs 94.0). That makes Flash-RL a reasonable default if you’re building an application where response time and GPU cost outweigh a few points of raw capability.
Both checkpoints come from the same “You Only RL Once” training approach: instead of training separate models for coding, agentic tool use, vision tasks, and cybersecurity, Xiaomi mixes all of these task types into a single reinforcement learning run using Group Relative Policy Optimization (GRPO) at large batch sizes. The idea is that skills transfer across domains, so a model trained on code agents and cybersecurity tasks together generalizes better to tool harnesses it’s never seen.
What hardware do you need to run it?
Because MiMo-V2.6-Flash-RL activates 15B parameters per token out of 309B total, it behaves computationally more like a mid-sized dense model at inference time, but the full weight set still has to sit in GPU memory (or be sharded across GPUs) since MoE routing can call on any of the 256 routed experts. The FP8 quantized weights reduce the memory footprint relative to full precision, but this is still a multi-GPU model, not something you run on a single consumer card.
The official SGLang command specifies --tp 8 --dp 2, meaning eight-way tensor parallelism combined with two-way data parallelism, which implies a setup with 16 GPU ranks total in the reference configuration. The vLLM command uses --tensor-parallel-size 4, a lighter footprint, though vLLM support may lag behind SGLang since Xiaomi notes that “stable vLLM may lag” and points users to a custom pre-built Docker image tagged vllm/vllm-openai:mimov25-cu129. In both cases, you’re looking at a multi-GPU server-class deployment, not a laptop or single-workstation setup.
How do you deploy it with SGLang?
Xiaomi recommends SGLang as the primary deployment path and publishes a dedicated cookbook for it. The reference Docker image is lmsysorg/sglang:latest. A representative serving command looks like this:
sglang serve \
--trust-remote-code \
--model-path XiaomiMiMo/MiMo-V2.6-Flash-RL \
--tp 8 \
--dp 2 \
--enable-dp-attention \
--enable-dp-lm-head \
--mm-enable-dp-encoder \
--mem-fraction-static 0.65 \
--chunked-prefill-size 16384 \
--speculative-algorithm EAGLE \
--speculative-num-steps 3 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 4 \
--enable-multi-layer-eagle \
--reasoning-parser mimo \
--tool-call-parser mimo \
--host 0.0.0.0 \
--port 30000Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
A few flags are worth understanding before you copy-paste this. --enable-dp-attention and --enable-dp-lm-head distribute attention and the language modeling head across data-parallel ranks, which matters for a model this size. --mm-enable-dp-encoder extends that same data-parallel treatment to the multimodal encoders, since this is an omnimodal model with separate vision and audio encoder stacks. The speculative decoding flags (--speculative-algorithm EAGLE and friends) turn on the model’s built-in 5-layer speculative decoder, which drafts multiple future tokens per forward pass for parallel verification, a technique that speeds up generation without changing output quality. Finally, --reasoning-parser mimo and --tool-call-parser mimo tell SGLang how to parse the model’s structured reasoning and tool-call output formats, which is necessary if you’re using this model in an agentic pipeline that expects structured function calls.
How do you deploy it with vLLM?
vLLM support follows a published recipe for the MiMo-V2.5 line, and Xiaomi notes that the stable public vLLM release can lag behind what’s needed for the newest MiMo checkpoints, so they provide a pinned Docker image (vllm/vllm-openai:mimov25-cu129) to avoid compatibility issues. The reference serving command is:
vllm serve XiaomiMiMo/MiMo-V2.6-Flash-RL \
--tensor-parallel-size 4 \
--trust-remote-code \
--gpu-memory-utilization 0.95 \
--max-model-len auto \
--reasoning-parser mimo \
--tool-call-parser mimo \
--enable-auto-tool-choice \
--generation-config vllmNote the --gpu-memory-utilization 0.95, which tells vLLM to aggressively use nearly all available GPU memory for the KV cache and model weights, a common setting for large models where you want maximum throughput and have already sized your hardware appropriately. Recommended sampling parameters for both backends are temperature=1.0 and top_p=0.95, which is a fairly high-temperature default suited to the model’s agentic and reasoning-heavy training.
Is it worth running MiMo-V2.6-Flash-RL yourself?
Whether self-hosting makes sense depends on your use case. If you already have access to a multi-GPU cluster and want full control over data, latency, and cost at scale, Flash-RL is a real option: its evaluation numbers are competitive with commercial models like Claude Opus 5, GPT-5.6 Sol, and Claude Fable 5 on several agent and cybersecurity benchmarks, even if it trails on others like Terminal Bench 4.0 and ExploitBench. If you don’t have that hardware on hand, Xiaomi offers the same model through hosted options: AI Studio, MiMo Code, MiMo Desktop, the Xiaomi MiMo API platform, and OpenRouter, all of which sidestep the GPU provisioning problem entirely.
For teams building coding agents, general-purpose automation agents, or multimodal pipelines that need long context windows (up to 1M tokens), Flash-RL’s efficiency profile, 15B active parameters against a 309B total, is the main selling point over the Pro-RL variant. You give up a small amount of benchmark performance in exchange for meaningfully lower inference cost per token, which matters a lot when you’re running agentic workloads that make many sequential calls.
Frequently Asked Questions
How many GPUs does MiMo-V2.6-Flash-RL need?
The official SGLang example uses 8-way tensor parallelism and 2-way data parallelism, implying a multi-GPU node setup. The vLLM example uses 4-way tensor parallelism. Exact GPU count and VRAM needed depend on your target context length and batch size, since the model supports contexts up to 1 million tokens.
What’s the difference between MiMo-V2.6-Flash-RL and MiMo-V2.6-Pro-RL?
Both come from the same reinforcement learning training pipeline, but Flash-RL is the smaller, faster-inference checkpoint. It trails Pro-RL by small margins on most benchmarks but is optimized for lower latency and cost.
Does MiMo-V2.6-Flash-RL support tool calling and reasoning traces?
Yes. Both the SGLang and vLLM deployment commands include --reasoning-parser mimo and --tool-call-parser mimo flags, which parse the model’s native reasoning and structured tool-call output formats for use in agentic applications.
Can I run MiMo-V2.6-Flash-RL without self-hosting?
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
Yes. Xiaomi offers the model through AI Studio, MiMo Code, Xiaomi MiMo Desktop, the Xiaomi MiMo Open Platform API, and OpenRouter, in addition to the open-weight release for self-hosted deployment.
What modalities does the model support?
MiMo-V2.6-Flash-RL is natively omnimodal, handling text, image, video, and audio through a shared architecture that includes a 681M-parameter vision encoder and separate audio tokenizer and patch encoder components.
