Kimi K3.1 Pro: Inside Moonshot AI's 2.8T Parameter Open Model
Kimi K3.1 Pro specs explained: 2.8T MoE parameters, 104B active, 1M context, and how it benchmarks against Claude and GPT.
What is Kimi K3.1 Pro?
Kimi K3.1 Pro is an open-weight, multimodal Mixture-of-Experts model from Moonshot AI, built on 2.8 trillion total parameters with 104 billion active per token. It’s the first release in the K3.1 series, post-trained from Kimi K3 with large-scale reinforcement learning across hundreds of downstream domains, and it ships with a 1-million-token context window. Moonshot positions it as an agentic model, meaning it’s tuned for coding, tool use, and multi-step tasks rather than just chat.
TL;DR
- Kimi K3.1 Pro uses a sparse MoE design with 2.8T total parameters but only 104B activated per forward pass, keeping inference costs closer to a much smaller dense model.
- The model swaps in Gated DeepSeek Sparse Attention (Gated DSA) layers alongside Kernelized Delta Attention (KDA), a change from Kimi K3’s full-attention Gated MLA layers, aimed at making the 1M-token context window more efficient to run.
- It ships natively multimodal with a 401M-parameter MoonViT-V2 vision encoder, so text and image inputs run through one model rather than a bolted-on adapter.
- Moonshot trained it with quantization-aware training for MXFP4 weights and MXFP8 activations, which lets the released weights run at lower precision without the usual post-hoc accuracy hit.
- On coding benchmarks it’s competitive with or ahead of Claude Opus 5.5 and GPT-6 Astra in several tests (Terminal-Bench 4.0, SWE Marathon, DeepSWE), though it trails on others like Program Bench and Terminal-Bench-Science.
- The model card reports scores against Claude Fable 5.1, GPT-6 Astra, Claude Opus 5.5, DeepSeek-V4.1-Flash, and GLM-5.3, giving builders a direct cross-vendor comparison rather than cherry-picked wins.
- It’s available as open weights on Hugging Face under a Kimi-specific license, alongside an API on Moonshot’s own platform.
Without shadow IT.
How big is Kimi K3.1 Pro, really?
The headline number is 2.8 trillion total parameters, but that figure is misleading if you’re thinking about compute cost. Like other MoE models, Kimi K3.1 Pro only activates a fraction of its parameters for any given token. Here that’s 104 billion, routed through 16 selected experts out of a pool of 896, plus 2 shared experts that always fire. The architecture spreads this across 93 layers, of which only 1 is a traditional dense layer; the rest are MoE blocks with a hidden dimension of 3072 per expert and a latent MoE dimension of 3584.
So in practical terms, you’re deploying something with the storage footprint of a 2.8T model but the per-token compute profile closer to a 100B-parameter dense model. That’s the standard MoE trade: more capacity for specialization, without a proportional hit to inference speed, at the cost of needing enough memory or disk to hold the full expert set.
What changed in the attention mechanism?
The most technical, and arguably most consequential, change from Kimi K3 is in how the model handles attention. K3.1 Pro’s 93 layers split into 69 KDA (Kernelized Delta Attention) layers and 24 Gated DSA (Gated DeepSeek Sparse Attention) layers. Moonshot’s model card explicitly states this replaces the full-attention Gated MLA layers used in K3, with the stated goal of making long-context modeling more efficient.
This matters because a 1,048,576-token context window is expensive to serve with full attention, where cost scales quadratically with sequence length. Sparse attention mechanisms like DSA restrict each token to attending over a selected subset of the sequence (here, a DSA index top-K of 2048) rather than every previous token. Combined with KDA’s linear-attention-style approach in the majority of layers, this is Moonshot’s answer to making million-token context windows usable rather than theoretical. The attention hidden dimension stays at 7168 across 96 heads, with the DSA index using 32 heads at a dimension of 128.
What does “natively multimodal” mean here?
Kimi K3.1 Pro handles text and image inputs through a single architecture rather than stitching a separate vision model onto a language model after the fact. The vision side runs on MoonViT-V2, a 401-million-parameter encoder, which is small relative to the 2.8T language backbone but handles the job of turning images into tokens the rest of the model can reason over. The model card lists its modalities as text and image (not audio or video), and Hugging Face tags the model under “image-text-to-text,” confirming vision input is a first-class, not experimental, capability.
For builders, this means tasks like reading charts, diagrams, screenshots, or CAD-style visual references can go through the same model and context window as text and code, without a separate multimodal pipeline.
How does Kimi K3.1 Pro compare to Claude Opus 5.5 and GPT-6 Astra?
Without shadow IT.
Moonshot’s own model card runs K3.1 Pro against Claude Fable 5.1, GPT-6 Astra, Claude Opus 5.5, DeepSeek-V4.1-Flash-1017, and GLM-5.3, all at their respective maximum reasoning-effort settings. The results are mixed rather than a clean sweep, which is itself notable for a vendor-published benchmark table.
In coding, K3.1 Pro posts 65.7 on Terminal-Bench 4.0, just behind Claude Opus 5.5’s 66.4 but well ahead of GLM-5.3 (41.8) and DeepSeek-V4.1-Flash (42.3). On SWE Marathon v1.1, a long-horizon software engineering test, K3.1 Pro leads the field at 51.7, ahead of Claude Fable 5.1 (45.6) and GPT-6 Astra (42.5). It also tops DeepSWE v1.1 at 75.5, narrowly ahead of Claude Opus 5.5 (74.2) and GPT-6 Astra (74.1).
But it’s not uniformly on top. On Terminal-Bench-Science, K3.1 Pro scores 38.1, well behind GPT-6 Astra’s 68.1 and Claude Opus 5.5’s 63.3. On Program Bench, it posts 31.5 against Claude Opus 5.5’s 65.0, a sizable gap. On BenchCAD without tool access, K3.1 Pro (72.8) runs close to Claude Opus 5.5 (73.0) and ahead of Claude Fable 5.1 (60.6), but with tools enabled, Claude Opus 5.5 (96.2) and GPT-6 Astra (95.9) edge slightly ahead of K3.1 Pro’s 92.7.
The pattern that emerges: K3.1 Pro is strong on agentic, long-horizon software engineering work and holds its own in CAD-style structured generation, but it’s inconsistent on narrower, specialized coding benchmarks like Terminal-Bench-Science and Program Bench. For agentic tool-use tasks like HLE-Full with tools, K3.1 Pro scores 70.0, ahead of Claude Fable 5.1’s 65.6, though the full agentic comparison table wasn’t complete in the released data.
Is Kimi K3.1 Pro worth deploying?
That depends on what you’re optimizing for. If the workload is long-horizon coding agents, SWE-style repository tasks, or anything that benefits from a 1-million-token context window, K3.1 Pro’s benchmark profile and architecture choices (sparse attention, MoE scaling) make it a reasonable fit, and it’s open-weight, which matters if self-hosting or fine-tuning control is a priority.
If the workload leans on narrower technical benchmarks like scientific reasoning in Terminal-Bench-Science or general programming correctness in Program Bench, the model card’s own numbers show Claude Opus 5.5 and GPT-6 Astra ahead by meaningful margins. There’s no single best model across every category in Moonshot’s own table, which is a more honest signal than most vendor benchmark releases.
The practical blocker for most teams will be infrastructure. Even with MXFP4/MXFP8 quantization-aware training reducing the deployment footprint, a 2.8T-parameter MoE model, shipped across 96 safetensors shard files, requires serious multi-GPU infrastructure to run locally. For teams without that capacity, the realistic path is Moonshot’s hosted API rather than self-hosting the open weights.
Frequently Asked Questions
How many parameters does Kimi K3.1 Pro actually use per token?
It has 2.8 trillion total parameters but activates only 104 billion per token, since it routes each token through 16 of 896 experts plus 2 shared experts that are always active.
What is Gated DSA and why does it matter?
Gated DSA (Gated DeepSeek Sparse Attention) is a sparse attention mechanism that replaces the full-attention layers used in the previous Kimi K3 model. It restricts each token’s attention to a limited subset of prior tokens (a top-K of 2048), which makes serving the 1-million-token context window computationally feasible.
Does Kimi K3.1 Pro support images, not just text?
Yes. It includes a 401-million-parameter MoonViT-V2 vision encoder built into the architecture, and Hugging Face lists it under the image-text-to-text pipeline, confirming native multimodal input.
How does Kimi K3.1 Pro compare to GLM-5.3?
Across the coding benchmarks in Moonshot’s model card, K3.1 Pro consistently outperforms GLM-5.3, for example 65.7 vs 41.8 on Terminal-Bench 4.0 and 77.1 vs 61.7 on Kimi’s internal Code Bench 2.0.
Can I run Kimi K3.1 Pro on my own hardware?
The weights are open and available on Hugging Face, split across 96 safetensor shard files, with quantization-aware training enabling MXFP4 weights and MXFP8 activations. Even so, a 2.8T-parameter MoE model requires substantial multi-GPU infrastructure, so self-hosting is realistic mainly for teams with datacenter-scale resources.
