Ornith 1.5 35B-A3B Benchmarks: How It Stacks Up Against Qwen3.6
Ornith 1.5 35B-A3B benchmark breakdown vs Qwen3.6-35B, Gemma 4-31B and Muse Glimmer-30B across SWE-bench, Terminal-Bench, HLE and agentic tests.
What is Ornith 1.5 35B-A3B and how does it compare to Qwen3.6?
Ornith 1.5 35B-A3B is a mixture-of-experts language model from Deep Reinforce that activates roughly 3 billion parameters per token out of a 35 billion total. According to its official model card, it outperforms the similarly sized Qwen3.6-35B-A3B across every published coding and agentic benchmark, and it beats larger dense models like Gemma 4-31B and Muse Glimmer-30B by wide margins on agentic coding tasks. On several agentic benchmarks it also closes much of the gap with Qwen3.5-397B, a model roughly eleven times its total size.
TL;DR
- Ornith 1.5 is a 35B-total, ~3B-active MoE model built by extending Ornith 1.0, which itself was developed on top of Qwen3.5 and Gemma4 with additional pretraining and post-training.
- The model’s headline feature is a joint self-improvement loop that generates its own training tasks, builds scaffolds to solve them, and optimizes rollouts through reinforcement learning (GRPO), rather than relying on fixed human-curated datasets.
- On SWE-bench Verified it scores 79, ahead of Qwen3.6-35B’s 73.4 and even ahead of the much larger Qwen3.5-397B’s 76.4.
- On Terminal-Bench 2.1 (Terminus-2 harness) it hits 67.8 versus 52.5 for Qwen3.6-35B, and on the Claude Code harness it scores 68.5 versus 49.2.
- The DeepSWE benchmark shows the starkest gap: Ornith 1.5 scores 22 while Qwen3.6-35B and Gemma 4-31B both score 0.
- On pure reasoning benchmarks like HLE and GPQA Diamond, Ornith 1.5 leads its size class but still trails the much larger Qwen3.5-397B, which scores notably higher on HLE with tools (48.3 vs Ornith’s 33.4).
- A community test running the model locally on a single A100 GPU showed it independently planning, building, and self-verifying a real-time price alert feature added to a live full-stack crypto dashboard app.
What benchmarks does Ornith 1.5 report, and what do the scores show?
The model card publishes results across three categories: coding, reasoning, and agentic tasks, comparing Ornith 1.5 35B-A3B against its own predecessor (Ornith 1.0 35B-A3B), Qwen3.6-35B-A3B, Gemma 4-31B, Muse Glimmer-30B, and the much larger Qwen3.5-397B.
In coding, the gaps are consistent and sometimes large:
- Terminal-Bench 2.1 (Terminus-2): 67.8 for Ornith 1.5 vs 64.2 (Ornith 1.0), 52.5 (Qwen3.6-35B), 42.1 (Gemma 4-31B), 51.7 (Muse Glimmer-30B), and 53.5 (Qwen3.5-397B).
- SWE-bench Verified: 79 vs 75.6, 73.4, 52, 76, and 76.4 respectively. Ornith 1.5 is the only model in the table to clear 79.
- SWE-bench Pro: 59.6, ahead of Qwen3.6-35B’s 49.5 and Gemma 4-31B’s 35.7.
- SWE-bench Multilingual: 71.4, again leading Qwen3.6-35B (67.2) and roughly matching Qwen3.5-397B (69.3).
- DeepSWE: 22 for Ornith 1.5 against 0 for both Ornith 1.0 and Qwen3.6-35B, and just 1 for Qwen3.5-397B. This is the widest relative margin in the whole table.
- Frontier-Bench v0.1: 5.1 vs 1.4 for Ornith 1.0, Qwen3.6-35B, and Qwen3.5-397B alike, all of which post identical scores.
- NL2Repo: 46.2, ahead of Qwen3.6-35B’s 29.4 and Qwen3.5-397B’s 36.8.
- SWE Atlas QnA: 39.8, more than double Qwen3.6-35B’s 15.5.
In reasoning, the pattern shifts. On HLE without tools, Ornith 1.5 scores 25.6, ahead of every same-size competitor but behind Qwen3.5-397B’s 28.7. With tools, it scores 33.4, still trailing Qwen3.5-397B’s 48.3 by a wide margin. On GPQA Diamond it scores 89.2, slightly ahead of Qwen3.5-397B’s 88.4 and comfortably ahead of Qwen3.6-35B’s 86.
In the agentic category, Ornith 1.5 scores 70.2 on MCP-Atlas, ahead of Qwen3.6-35B (62.8) and Gemma 4-31B (55) but behind Muse Glimmer-30B (75.5) and Qwen3.5-397B (72.3). On Toolathlon-Verified it scores 48.7, edging out Qwen3.6-35B’s 41.7.
Why does the self-improvement training loop matter for these results?
Ornith 1.5’s model card attributes its performance to expanding the self-improvement loop beyond what Ornith 1.0 used. Instead of optimizing only scaffold construction and rollout generation against a fixed set of human-written tasks, Ornith 1.5 jointly optimizes three things at once: the tasks themselves, the scaffolds used to attempt them, and the rollouts that solve them.
Concretely, the loop works like this: the system builds a task and a scaffold for tackling it, generates multiple candidate solution rollouts, and scores those rollouts based on how well they perform. Reinforcement learning, specifically GRPO (Group Relative Policy Optimization), then feeds that reward signal back to improve both the scaffold and the solutions together. Because the model is generating its own training curriculum rather than depending entirely on hand-curated benchmarks, it can in principle keep discovering new task types and better strategies without a human bottleneck.
This design shows up in the DeepSWE numbers, where Ornith 1.5 scores 22 against a flat 0 for its closest same-size rival. It also explains the qualitative behavior seen when the model was tested by a community reviewer: given an open-ended goal on a live full-stack application (a crypto price dashboard with a FastAPI backend, WebSocket price feed, and Redis-backed history running across Docker containers), the model planned its own approach, wrote code, ran its own tests, checked results, and iterated before declaring the task done, without being told what steps to follow.
How does it perform on agentic and tool-use tasks in practice?
The benchmark table’s agentic section (MCP-Atlas, Toolathlon-Verified) is only partially populated for some competitors, but the pattern of same-size dominance holds where data exists. Ornith 1.5’s advantage over Qwen3.6-35B is consistent, if narrower, than its coding-benchmark lead.
A hands-on test captured this gap in a real environment. Given the instruction to “build a real-time price alert feature end-to-end and prove it works before it stops,” with no checklist or implementation details, the model explored the existing codebase, planned a design, broke it into implementation chunks, wrote code, identified that a required network port was already occupied (by the inference server itself), adjusted, wrote and ran tests, and confirmed pass/fail status before finishing. The resulting feature, live price alerts with editable thresholds on a running crypto tracker, worked when checked in a browser afterward.
Is Ornith 1.5 worth running locally?
For teams evaluating open-weight coding and agentic models, the MoE architecture is the key practical detail: only about 3 billion of the 35 billion total parameters activate per token, which keeps inference costs closer to a much smaller dense model while benchmark scores compete with far larger systems. In one demonstration, the model ran on a single Nvidia A100 GPU (80GB VRAM) using vLLM for serving, consuming close to 74GB including KV cache, which can be reduced by lowering the KV cache allocation.
Whether it’s “worth it” depends on the use case. The benchmark card shows Ornith 1.5 leading its weight class on nearly every coding and agentic metric, including a notable jump on DeepSWE and Frontier-Bench where competitors scored near zero. But on pure reasoning under Humanity’s Last Exam (HLE) with tool use, it still trails the far larger Qwen3.5-397B by double digits. For coding-heavy and agentic workflows where compute efficiency matters, it’s a strong candidate. For pure reasoning tasks where model scale still seems to matter, larger dense or MoE models retain an edge.
Frequently Asked Questions
What does “35B-A3B” mean in Ornith 1.5’s name?
It denotes a mixture-of-experts model with 35 billion total parameters, of which only about 3 billion (“A3B,” active 3B) are activated for any given token, which keeps inference cheaper than a dense model of the same total size.
How does Ornith 1.5 differ from Ornith 1.0?
Ornith 1.0 was built on top of Qwen3.5 and Gemma4 with continued pretraining, mid-training, and post-training, and optimized scaffold construction and rollout generation against fixed tasks. Ornith 1.5 extends this by also letting the model generate and optimize its own training tasks, jointly training task generation, scaffolding, and rollouts together.
Does Ornith 1.5 beat larger models like Qwen3.5-397B?
Not universally. It matches or edges out Qwen3.5-397B on some coding benchmarks like SWE-bench Verified (79 vs 76.4) and GPQA Diamond (89.2 vs 88.4), but trails significantly on HLE with tools (33.4 vs 48.3) and on MCP-Atlas (70.2 vs 72.3).
What hardware is needed to run Ornith 1.5 35B-A3B locally?
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
A demonstration used a single Nvidia A100 GPU with 80GB of VRAM, served via vLLM, with the model plus KV cache consuming close to 74GB. Reducing the KV cache allocation can lower memory usage further.
Is Ornith 1.5 a general-purpose model or specialized for coding?
Unlike Ornith 1.0, which was not designed as general-purpose, Ornith 1.5 is built for broader intelligence across reasoning, agentic, and coding tasks, based on its model card and behavior observed in multilingual and creative reasoning tests alongside its coding benchmarks.
