Raon-1 mini: KRAFTON's Korean-Focused Open MoE Model Explained
KRAFTON AI's Raon-1 mini is a 30B-A4B MoE model with hybrid attention, built for Korean conversation and agentic tasks and workloads.
What is Raon-1 mini?
Raon-1 mini is an open-weight Mixture-of-Experts language model built by KRAFTON AI, the South Korean game studio behind PUBG, through its AI research arm. It has 28.8 billion total parameters but activates only 4.3 billion per token (commonly labeled 30B-A4B), trained from scratch rather than fine-tuned from an existing base. The model targets Korean conversation and agentic workloads, is released under Apache 2.0, and ships as a preview, meaning specs and benchmark numbers may shift before a final version lands.
TL;DR
- KRAFTON AI, better known for PUBG, built Raon-1 mini as a from-scratch 30B-A4B MoE model rather than a fine-tune of an existing open model.
- The architecture mixes 18 linear attention layers with 7 full attention layers, using a custom method called Raon Delta Attention to handle long context more efficiently than standard transformer attention.
- A technique called Raon-mHC reworks DeepSeek’s manifold-constrained hyper-connections to cut training cost while keeping the benefits of multi-stream residual connections.
- Benchmark tables on the model card show Raon-1 mini beating or matching Qwen3.6-35B-A3B and Gemma 4 26B-A4B on reasoning tasks like GPQA Diamond and AIME 2026, while trailing on broad knowledge tests like MMLU Pro.
- The model supports context windows up to 262,144 tokens and needs only about 8 KiB of KV cache per token in BF16, because most layers don’t keep a traditional KV cache at all.
- It’s quantized in MXFP8 and NVFP4, with the NVFP4 version small enough to run inference on a single RTX 5090 with 32 GB of VRAM.
- KRAFTON AI also released a matching DFlash2 draft model for speculative decoding, aimed at making inference faster in practice, not just on paper.
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
Why is a game studio building LLMs?
KRAFTON is primarily known as a video game company, but it has run an internal AI research group for several years, investing in generative AI for both game development tools and general-purpose language models. Raon-1 mini is part of what the company calls the “Raon” open model family, and the choice to train from scratch (rather than adapt Qwen, Llama, or another existing base) signals an attempt to build real in-house modeling expertise rather than just fine-tune someone else’s work. That’s a heavier lift than most labs attempt, since pretraining a 28.8B-parameter MoE model from zero requires substantial compute and data pipeline investment. The payoff, according to KRAFTON’s own benchmark claims, is a model that performs competitively against established open MoE models while specializing in Korean language tasks that larger multilingual models often handle less precisely.
How does Raon Delta Attention work?
Raon Delta Attention (RDA) is a modification of Kimi Delta Attention (KDA), a linear attention mechanism originally developed for Moonshot AI’s Kimi models. Standard transformer attention computes relationships between every token and every other token, which gets expensive as context grows. Linear attention methods instead maintain a compressed, fixed-size “state” that gets updated as new tokens arrive, trading some precision for much lower memory and compute cost at long context lengths.
KDA uses “delta” updates, where the model decides what to erase from its running state and what to write into it. KRAFTON’s change decouples the erase and write gates that KDA ties together, letting the model control forgetting and updating independently. According to the model card, this decoupling improves long-context performance compared to the original KDA design. In Raon-1 mini, 18 of the 25 total layers use RDA with this fixed-size state, while only 7 layers use full “Gated MLA” (multi-head latent attention) with an actual KV cache. That hybrid ratio, roughly 3-to-1 linear-to-full attention, is what lets the model support a 262,144-token context window without the memory cost scaling the way it would in a pure transformer.
What does Raon-mHC do differently?
The second named technique, Raon-mHC, builds on “manifold-constrained hyper-connections” (mHC), a method DeepSeek introduced for improving how information flows through a network’s residual connections. Instead of a single residual stream carrying signal from one layer to the next, hyper-connections maintain multiple parallel streams (Raon-1 mini uses four), which can improve gradient flow and training stability in deep networks.
The tradeoff is that multi-stream residual connections add computational overhead. KRAFTON’s Raon-mHC is described as a way to reduce that cost relative to DeepSeek’s original formulation, which in turn speeds up training. For a lab training a model from scratch, shaving compute off every training step compounds significantly over a full training run, so this kind of architectural efficiency work matters more here than it would for a fine-tuning project.
How does it compare to Qwen3.6-35B-A3B and Gemma 4?
Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
KRAFTON’s own benchmark table positions Raon-1 mini against four comparison models: Qwen3.6-35B-A3B and Gemma 4 26B-A4B (both MoE models), plus Qwen3.8-27B and Muse Glimmer 30B (dense models) shown for reference.
On reasoning benchmarks, Raon-1 mini comes out ahead or roughly even with the MoE competitors. It scores 84.3 on GPQA Diamond versus 83.2 for Qwen3.6-35B-A3B and 80.7 for Gemma 4, and 93.3 on AIME 2026 versus 92.9 and 87.1 respectively. On HMMT Feb 2026, it’s close behind Qwen3.6-35B-A3B (84.1 versus 84.5) but well ahead of Gemma 4 (79.9).
On broader knowledge benchmarks, the picture flips. Raon-1 mini scores 76.5 on MMLU Pro, behind Qwen3.6-35B-A3B’s 85.9 and Gemma 4’s 82.9. On AA-Omniscience, a benchmark that penalizes confidently wrong answers, Raon-1 mini posts a positive 3.5 score while all four comparison models score negative, meaning they lose more points for overconfident mistakes than they gain for correct answers. That pattern suggests Raon-1 mini is tuned to be more conservative or calibrated in situations outside its strongest domains, which matters for agentic use cases where a model confidently hallucinating a tool call or a fact can break a pipeline. KRAFTON also claims the model leads on its instruction-following and search benchmarks, though the full tables for those weren’t fully detailed in the available model card text.
What hardware does it need to run?
Raon-1 mini is distributed in two quantization formats: MXFP8 and NVFP4. The NVFP4 version is small enough to run inference on a single NVIDIA RTX 5090 with 32 GB of VRAM, which is notable for a model with 28.8B total parameters, since unquantized or lightly quantized MoE models of that size typically need multiple GPUs or a workstation-class card with far more memory. The hybrid attention design helps here too: because most layers don’t maintain a full KV cache, the memory overhead for long-context inference stays relatively low (around 8 KiB per token in BF16, concentrated in just the 7 full-attention layers).
For throughput, KRAFTON also released a DFlash2 draft model specifically for speculative decoding, a technique where a small, fast “draft” model proposes several tokens ahead and the larger model verifies them in a batch, speeding up generation without changing output quality. This is paired with the main checkpoint as part of the release rather than being a separate add-on users have to source themselves.
Is Raon-1 mini useful outside Korean language tasks?
The model card describes training on seven languages: Korean, English, Japanese, Chinese, Spanish, French, and Thai, with the strongest performance in Korean and “solid” capability in English. For agentic workloads, it supports tool calling and lists compatibility with Hermes Agent, Pi, and OpenCode, which suggests KRAFTON built it with developer tooling and automation pipelines in mind, not just conversational chat. Whether it’s a good general-purpose pick for non-Korean, non-English use cases is unclear from the available documentation, since the other four languages aren’t benchmarked in the released comparison tables.
Frequently Asked Questions
What does “30B-A4B” mean for Raon-1 mini?
It describes a Mixture-of-Experts model with roughly 30 billion total parameters distributed across many experts, of which only about 4 billion are “activated,” or actually used, for any given token. Raon-1 mini’s exact figures are 28.8B total and 4.3B activated, using 6 routed experts plus 1 shared expert out of 128 total per token.
Is Raon-1 mini open source?
Yes. It’s released under the Apache 2.0 license, which permits commercial use, modification, and redistribution, and the weights are hosted on Hugging Face under the KRAFTON organization account.
How is Raon Delta Attention different from standard attention?
Standard (full) attention compares every token to every other token, which scales poorly with context length. Raon Delta Attention is a linear attention method that maintains a fixed-size running state instead, updating it with separate “erase” and “write” operations. This keeps memory and compute costs much lower at long context lengths, at the cost of some precision compared to full attention.
Can I run Raon-1 mini on consumer hardware?
The NVFP4-quantized version is built to run on a single NVIDIA RTX 5090 with 32 GB of VRAM, which puts it within reach of high-end consumer or prosumer setups rather than requiring a multi-GPU server.
How does Raon-1 mini compare to Qwen and Gemma models?
Based on KRAFTON’s published benchmark table, it beats or closely matches Qwen3.6-35B-A3B and Gemma 4 26B-A4B on reasoning benchmarks like GPQA Diamond and AIME 2026, but trails both on the broader MMLU Pro knowledge benchmark.
