Qwen3.8-2.4T-A95B: Alibaba's Open-Weight Qwen-Max Flagship Explained
Alibaba open-sourced Qwen3.8-2.4T-A95B, a 2.4T-parameter MoE model with 95B active params and 262K context. Here's what's inside it.
Drafted with Claude from source material, checked by automated verification, and reviewed before release. How we make these.
What is Qwen3.8-2.4T-A95B?
Qwen3.8-2.4T-A95B is Alibaba’s newest open-weight language model, and the first time the company has released a Qwen-Max-class model to the public. It’s a mixture-of-experts (MoE) model with 2.4 trillion total parameters, of which 95 billion are active per forward pass. It natively supports 262,144 tokens of context, extensible to just over 1 million, and it’s available now on Hugging Face under the Qwen organization, compatible with vLLM, SGLang, and other standard inference stacks.
TL;DR
- Qwen3.8-2.4T-A95B is Alibaba’s first open release of a model built on the same tier as its proprietary Qwen-Max line, marking a shift in how much capability the company is willing to put into open weights.
- The model uses a mixture-of-experts architecture with 2.4T total parameters but only 95B active per token, keeping inference costs closer to a mid-size dense model despite the massive total parameter count.
- Context length runs to 262,144 tokens natively, with extension up to roughly 1,010,000 tokens for long-document and long-horizon agentic work.
- On released benchmarks, Qwen3.8-Max (the hosted version of this model) beats its predecessor Qwen3.7-Max by wide margins across nearly every coding and agent benchmark listed.
- Against closed frontier models like Opus 4.8, Fable 5, and GPT 5.6 Sol, Qwen3.8-Max is competitive on several benchmarks and leads on a few, including PaperBench and FrontierSWE, while trailing on others like DeepSWE 1.1.
- The architecture mixes Gated DeltaNet linear attention with regular Gated Attention in a repeating pattern, a design choice aimed at balancing long-context efficiency with retrieval quality.
- A hosted version called Qwen3.8-Max adds vision input, non-thinking mode, a 1M token default context window, and built-in tools through Alibaba’s Qwen Cloud service.
What’s inside the architecture?
Qwen3.8-2.4T-A95B builds on the architectural foundation Alibaba introduced with Qwen3.5. The model has 92 layers and a hidden dimension of 8192, with a padded token embedding size of 248,320.
The layer structure follows a repeating pattern: 23 blocks, each containing three units of Gated DeltaNet followed by MoE, followed by one unit of Gated Attention followed by MoE. This hybrid approach mixes linear attention (Gated DeltaNet, with 128 heads for V and 16 for QK, head dimension 128) with standard gated attention (64 query heads, 4 key/value heads, head dimension 256, rotary embedding dimension 64). The idea behind this kind of hybrid is to get most of the compute benefits of linear attention for long sequences while still leaning on full attention periodically to preserve retrieval accuracy over long contexts.
The MoE layer itself routes across 512 experts, activating 10 routed experts plus 1 shared expert per token, with each expert carrying an intermediate dimension of 2048. That routing setup is what keeps the active parameter count at 95B despite the 2.4T total. The model also uses multi-token prediction (MTP), trained across multiple steps, which typically speeds up inference by letting the model draft more than one token per forward pass.
How does Qwen3.8 compare to Opus 4.8, Fable 5, and GPT 5.6 Sol?
Alibaba’s own benchmark table pits the hosted Qwen3.8-Max against Anthropic’s Opus 4.8, an assistant referred to as Fable 5, and GPT 5.6 Sol (max), alongside the previous generation Qwen3.7-Max.
On coding-agent benchmarks, results are mixed but generally strong for Qwen3.8-Max. It scores 86.6 on Terminal Bench 2.1, ahead of Opus 4.8 (84.6) and Fable 5 (84.6) but behind GPT 5.6 Sol (88.8). On SWE-bench Pro it posts 67.7, trailing Fable 5’s 80.0 but ahead of GPT 5.6 Sol’s 64.6. On DeepSWE 1.1, Qwen3.8-Max scores 56.6, behind all three closed competitors (59.0 to 73.0), showing this remains an area where the open model lags.
Where Qwen3.8-Max stands out is on PaperBench, where it scores 93.0, the highest in the table, ahead of GPT 5.6 Sol (90.5), Fable 5 (88.8), and Opus 4.8 (80.3). It also leads FrontierSWE at 73.5, a large jump over Opus 4.8’s 70.0, though still behind Fable 5’s 88.8. On QwenSWEBench and QwenQoderBench (Alibaba’s own internal evals), Qwen3.8-Max scores competitively at 80.7 and 58.4 respectively, generally ahead of GPT 5.6 Sol but behind Fable 5.
Compared to its direct predecessor, Qwen3.7-Max, the jump is dramatic across the board. Terminal Bench 2.1 goes from 74.5 to 86.6. DeepSWE 1.1 nearly triples, from 21.6 to 56.6. FrontierSWE almost doubles, from 40.7 to 73.5. That generational gap is arguably the more important story here: Alibaba isn’t just releasing a bigger model, it’s releasing one that closes much of the gap to the top closed models in a single version bump.
What does the parameter count actually mean for running it?
The “2.4T-A95B” naming tells you the two numbers that matter most for anyone planning to deploy this model: 2.4 trillion total parameters and 95 billion activated parameters per token. In an MoE model, only a subset of experts fire for any given token, so the 95B figure is a better proxy for per-token compute cost than the 2.4T headline number.
That said, total parameter count still matters enormously for memory. All 2.4T parameters, or a quantized version of them, need to sit somewhere accessible during inference, even though only a fraction are used per forward pass. This is the classic MoE tradeoff: cheaper compute per token, but a large memory footprint to hold the full expert bank. The model ships across 213 safetensors shards on Hugging Face, an indication of just how much storage the full-precision weights require. Running the model at scale realistically means multi-GPU or multi-node serving infrastructure, which is why Alibaba positions its own Qwen Cloud API as the practical option for most users rather than self-hosting.
Is Qwen3.8-2.4T-A95B worth using over the hosted Qwen3.8-Max?
The Hugging Face repository hosts the raw post-trained weights in Transformers format, meant for teams that want to self-host using vLLM, SGLang, or similar inference engines. This gives full control over deployment, fine-tuning, and data handling, which matters for anyone with compliance or latency requirements that rule out a third-party API.
The hosted Qwen3.8-Max version, served through Qwen Cloud, adds features not present in the raw open weights: vision input support, a non-thinking mode for faster responses when deep reasoning isn’t needed, a 1 million token default context window (versus 262K native for the open weights), and built-in tools. For most developers who don’t need to manage their own GPU infrastructure, the hosted version is the more practical entry point, while the open weights matter most for teams doing custom fine-tuning, research, or on-premises deployment.
Why does this release matter for the open-model ecosystem?
Alibaba explicitly frames this as the first time a Qwen-Max-class model, previously reserved for the company’s flagship paid API tier, has been released with open weights. Prior open Qwen releases (the 3.5 and 3.6 series) were positioned a notch below the Max tier in capability. Putting a model that benchmarks competitively against Opus 4.8 and GPT 5.6 Sol into open release changes the calculus for teams evaluating whether to build on closed frontier APIs versus self-hosted alternatives, particularly for agentic coding and long-context workloads where the benchmark gains over Qwen3.7-Max are largest.
Frequently Asked Questions
What does “A95B” mean in the model name?
It refers to the number of activated parameters per token: 95 billion. The “2.4T” refers to the total parameter count across all experts in the mixture-of-experts architecture. Only a subset of those total parameters actually process each token.
How long a context window does Qwen3.8 support?
The open-weight model natively supports 262,144 tokens and can be extended up to approximately 1,010,000 tokens. The hosted Qwen3.8-Max service defaults to a 1 million token context window.
Is Qwen3.8-2.4T-A95B better than GPT 5.6 Sol or Opus 4.8?
It depends on the benchmark. Qwen3.8-Max leads on PaperBench and FrontierSWE in Alibaba’s released comparisons, but trails on others like DeepSWE 1.1 and SWE-bench Pro. It’s competitive rather than uniformly superior across the board.
Can I run this model on a single GPU?
Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
Given the 2.4T total parameter count spread across 213 safetensors shards, running the full model requires substantial multi-GPU or multi-node infrastructure. Only the 95B active parameters compute per token, but the full expert bank still needs to be held in memory or across a distributed setup.
What’s the difference between the open weights and Qwen3.8-Max?
The open-weight release on Hugging Face contains the post-trained model for self-hosting via vLLM or SGLang. Qwen3.8-Max is Alibaba’s managed API version, which adds vision input, non-thinking mode, a larger default context window, and built-in tools through Qwen Cloud.
