Xing4.0-29B-A4B: China Telecom's Ascend-Trained MoE Model Explained
Xing4.0-29B-A4B is a 29B MoE model trained fully on Ascend NPUs, with 256K context and coding scores beating Gemma4 and Qwen3.6 on several benchmarks.
What is Xing4.0-29B-A4B?
Xing4.0-29B-A4B is an open-weight mixture-of-experts (MoE) language model released by China Telecom Artificial Intelligence Technology Co., Ltd., under the Xing series (previously known as TeleChat). It has 29 billion total parameters but activates only 4 billion per token, routing across 64 experts (4 active plus 1 shared per token). The model natively handles a 256K token context window, extensible to 512K, and is released under an Apache 2.0 license on Hugging Face. What sets it apart isn’t just the architecture, it’s the training stack: this is the first model at this scale trained entirely on Huawei’s Ascend NPU platform using the MindSpore framework, rather than Nvidia GPUs.
TL;DR
- Sparse activation means Xing4.0-29B-A4B has 29B total parameters but only 4B active per token, keeping inference costs closer to a much smaller dense model while retaining a larger knowledge base.
- Ascend-native training makes this the first model of its size trained end-to-end on Huawei’s Ascend 910C NPUs with the MindSpore/MindFormers stack instead of the usual Nvidia-CUDA pipeline.
- Training throughput improved by roughly 96% over out-of-the-box performance through MoE communication optimization, selective recomputation, automatic graph-operator fusion, and custom Ascend C fused operators.
- Coding and agent benchmarks are the model’s strongest area, with a Terminal-Bench 2.1 score of 57.50 and a Claw-Eval score of 76.55, both ahead of Gemma4-26B-A4B and Qwen3.6-35B-A3B in the model’s own reported figures.
- General reasoning benchmarks are more mixed: Qwen3.6-35B-A3B edges it out on AIME2026, SWE-bench Verified, and Tau3-Bench, while Gemma4-26B-A4B leads on IFBench and AA.LCR.
- Framework compatibility covers Transformers, vLLM, SGLang, and KTransformers for inference, plus LLaMA-Factory and MindFormers for fine-tuning, with adapters for agent frameworks like OpenCode, Claude Code, OpenClaw, and Hermes.
- Architecture choices (mHC, MLA, and MTP) target multi-step planning and tool calling, which lines up with the model’s benchmark strength in agentic coding tasks over pure knowledge or instruction-following tests.
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
How does the architecture work?
Xing4.0-29B-A4B uses a combination of three architectural components the model card refers to as mHC, MLA (multi-head latent attention), and MTP (multi-token prediction). Together these are described as supporting multi-step planning, tool calling, and long reasoning chains, which is a fairly clear signal that the model was built with agentic coding and tool-use workloads in mind rather than pure chat.
The model has 40 layers, a hidden size of 3584, and a dense FFN intermediate size of 9216. On the MoE side, each expert has an intermediate size of 1024, and the model routes each token to 4 of 64 routed experts plus 1 shared expert. This is a fairly standard modern MoE layout (similar in spirit to DeepSeek-style and Qwen-style MoE designs), where a large expert pool lets the model specialize without inflating the active compute per token.
The 256K native context length (with room to extend to 512K) is large enough for long codebases, multi-file diffs, or extended agent trajectories without aggressive chunking.
Why train on Ascend NPUs instead of Nvidia GPUs?
The most unusual part of this release isn’t the benchmark numbers, it’s the training infrastructure. China Telecom trained Xing4.0-29B-A4B entirely on Huawei’s Ascend 910C clusters using MindSpore and MindFormers, rather than the Nvidia GPU and CUDA stack that underlies nearly every other major open-weight LLM release. The model card describes this as the first model of this scale to be trained end-to-end on Ascend hardware.
Getting a MoE model trained efficiently on non-Nvidia silicon is a nontrivial engineering problem: MoE architectures are unusually sensitive to communication overhead (routing tokens to experts across devices), and framework-level operator support tends to lag behind CUDA-first tooling. The model card lists several specific optimizations that made this practical: fine-grained MoE communication optimization, selective recomputation, DVM automatic graph-operator fusion, and custom Ascend C fused operators for the mHC component. Combined, China Telecom reports these changes improved training throughput by about 96% over the unoptimized baseline on the same hardware.
For readers tracking the broader chip and export-control story, this release is a concrete data point: large MoE models can now be trained competitively outside the Nvidia ecosystem, at least at this parameter scale.
How does it perform against Gemma4 and Qwen3.6?
The model card publishes benchmark comparisons against Gemma4-26B-A4B and Qwen3.6-35B-A3B, two similarly sized MoE models. The results are not a clean sweep in either direction, but a few patterns stand out.
Xing4.0-29B-A4B leads on agentic and coding-adjacent benchmarks:
- Claw-Eval: 76.55 vs. 71.49 (Gemma4) and 74.54 (Qwen3.6)
- Terminal-Bench 2.1: 57.50 vs. 30.00 (Gemma4) and 51.50 (Qwen3.6)
- DeepresearchBII: 60.80 vs. 39.30 (Gemma4) and 59.70 (Qwen3.6)
It’s competitive but slightly behind Qwen3.6 on other coding and agent-tool benchmarks:
- SWE-bench Verified: 75.00 vs. 76.00 (Qwen3.6), both well ahead of Gemma4’s 53.00
- SWE-bench Multilingual: 66.00 vs. 67.20 (Qwen3.6) and 51.00 (Gemma4)
- Tau3-Bench: 64.63 vs. 67.20 (Qwen3.6), ahead of Gemma4’s 58.90
And it trails on more general reasoning and instruction-following tests:
- IFBench: 69.67 vs. 72.67 (Gemma4), ahead of Qwen3.6’s 65.50
- AIME2026: 90.00 vs. 92.70 (Qwen3.6) and 88.30 (Gemma4)
- AA.LCR: 61.00 vs. 66.00 (Gemma4) and 62.00 (Qwen3.6)
The overall pattern: Xing4.0-29B-A4B is strongest on terminal and tool-use style agent benchmarks (Terminal-Bench, Claw-Eval, DeepresearchBII), roughly tied with Qwen3.6 on software-engineering benchmarks (SWE-bench Verified and Multilingual), and somewhat weaker on pure math and instruction-following benchmarks (AIME2026, IFBench, AA.LCR) where Gemma4 or Qwen3.6 take the lead. That’s consistent with the architecture’s stated focus on agent-oriented, multi-step tool use rather than general knowledge or strict instruction compliance.
Is Xing4.0-29B-A4B worth using for coding agents?
For teams building coding agents or terminal-automation tools, the benchmark profile is the main selling point. Terminal-Bench 2.1 and Claw-Eval specifically test an agent’s ability to operate tools and complete multi-step tasks in realistic environments, and Xing4.0-29B-A4B’s scores there (57.50 and 76.55) are notably higher than Gemma4’s and modestly ahead of Qwen3.6’s. SWE-bench Verified and SWE-bench Multilingual, which test actual code-fixing ability, put it in a near-tie with Qwen3.6-35B-A3B, both clearly ahead of Gemma4-26B-A4B.
Because only 4B parameters activate per token, inference cost and latency should sit closer to a small dense model than a 29B one, which matters for agent loops that make many sequential calls. The recommended generation settings differ by use case: temperature 1.0 / top_p 0.95 for general reasoning, and temperature 0.8 / top_p 0.95 for coding and agent tasks, both with a repetition penalty of 1.05.
The model card lists explicit adaptation work for OpenCode, Claude Code, OpenClaw, and Hermes agent frameworks, plus support for vLLM, SGLang, and KTransformers for serving, which lowers the integration cost for teams already using those tools.
Where it’s a weaker fit: workloads that lean heavily on strict instruction-following (IFBench) or long-context retrieval and citation accuracy (AA.LCR), where Gemma4 currently scores higher. Math-heavy reasoning (AIME2026) also favors Qwen3.6 by a few points.
Frequently Asked Questions
What does “29B-A4B” mean in the model name?
It describes the MoE structure: 29B total parameters exist in the model, but only 4B are activated for any given token, since the router selects a small subset of experts per token instead of running the full network.
What license is Xing4.0-29B-A4B released under?
It’s released under the Apache 2.0 license, making it usable for commercial applications without the restrictions some other open-weight licenses impose.
What is Ascend NPU training and why does it matter?
Ascend NPUs are Huawei’s AI accelerator chips, used here with the MindSpore/MindFormers software stack instead of Nvidia GPUs and CUDA. Training a MoE model of this scale entirely on Ascend hardware demonstrates that large-scale LLM training is viable outside the dominant Nvidia ecosystem, which matters for regions or organizations facing GPU export restrictions.
How does Xing4.0-29B-A4B compare to Qwen3.6-35B-A3B overall?
It’s close on coding benchmarks (SWE-bench Verified: 75.00 vs. 76.00) and ahead on agentic/terminal benchmarks (Terminal-Bench 2.1: 57.50 vs. 51.50), but behind on math and reasoning benchmarks like AIME2026 (90.00 vs. 92.70) and AA.LCR (61.00 vs. 62.00).
What frameworks support Xing4.0-29B-A4B?
For inference and deployment: Hugging Face Transformers, vLLM, SGLang, and KTransformers. For fine-tuning: LLaMA-Factory and MindFormers. It also includes format alignment for agent frameworks such as OpenCode, Claude Code, OpenClaw, and Hermes.
