How to Run Ornith-1.5-9B Locally with vLLM or SGLang

What is Ornith-1.5-9B?

Ornith-1.5-9B is the smallest model in the Ornith-1.5 family, a 9B dense reasoning model built on continued pretraining and post-training from a Qwen3.5/Gemma4-derived base (Ornith-1.0), then extended with a self-improvement training loop that jointly optimizes task generation, agent scaffolds, and reinforcement learning rollouts. It’s designed for single-GPU deployment and has a quantized mobile variant for edge devices. On released benchmarks it scores competitively against much larger models: 70.6 on SWE-bench Verified, 46.2 on Terminal-Bench 2.1, and 86.4 on GPQA Diamond, putting it ahead of Qwen3.5-9B across most reported categories and close to Qwen3.6-35B-A3B despite roughly a quarter of the active parameters.

TL;DR

Ornith-1.5-9B is a 9B dense reasoning model that fits on a single 80GB GPU, making it a realistic choice for teams that can’t run 30B+ mixture-of-experts models.
Benchmarks published by Ornith AI show it beating Qwen3.5-9B on nearly every coding, reasoning, and agentic metric, and trading blows with the much larger Qwen3.6-35B-A3B.
The model card notes it was evaluated with vLLM’s reasoning_content key, which means production serving needs a reasoning-aware inference stack rather than a plain chat completion setup.
Both vLLM and SGLang support the OpenAI-compatible serving pattern needed here, including reasoning parsers and tool-call parsers for agentic workloads.
The model uses a custom Qwen-derived chat template (chat_template.jinja), and the benchmark scripts explicitly modify Harbor and Claude Code harnesses to align with it, a sign that template mismatches are a real failure mode.
A 262K context window is achievable through RoPE scaling, which matters for long-document agentic tasks like SWE-bench Pro or WideSearch-style browsing.
Because it’s dense (not MoE), memory and compute scale predictably with sequence length, which simplifies capacity planning compared to sparse models.

One coffee. One working app.

You bring the idea. Remy manages the project.

WHILE YOU WERE AWAY

✓Designed the data model

✓Picked an auth scheme — sessions + RBAC

✓Wired up Stripe checkout

✓Deployed to production

Live at yourapp.msagent.ai

Why does the serving stack matter for a reasoning model?

Ornith-1.5-9B isn’t a plain instruct model. It’s trained with reinforcement learning over multi-step rollouts, and its outputs separate “thinking” content from the final answer. The model card specifically calls out that its Terminal-Bench 2.1 evaluations required modifying the Harbor test harness “to align with vLLM’s reasoning_content key.” That’s a strong hint about how the model expects to be served: an inference server that understands reasoning tokens as a distinct field, not just raw text appended to the response.

If you serve this model through a naive text-completion endpoint, you’ll get the reasoning trace mixed into the final answer, breaking downstream parsing for anything expecting clean structured output (tool calls, JSON, code blocks). This is exactly the gap that reasoning parsers in vLLM and SGLang are built to close.

How do you set up vLLM for Ornith-1.5-9B?

The general pattern for serving a reasoning-capable open-weight model on vLLM looks like this:

vllm serve ornith-ai/Ornith-1.5-9B \
  --reasoning-parser qwen3 \
  --tool-call-parser hermes \
  --enable-auto-tool-choice \
  --rope-scaling '{"rope_type": "yarn", "factor": 4.0, "original_max_position_embeddings": 65536}' \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90

A few things to check before you lock in flags:

Reasoning parser. Since Ornith-1.5 is built on a Qwen-lineage base and the model card explicitly references vLLM’s reasoning_content field, start with vLLM’s Qwen-family reasoning parser and validate that thinking tokens land in reasoning_content rather than content.
Chat template. The model ships its own chat_template.jinja, and the maintainers note it had to be adjusted “to ensure consistency between training and inference.” Don’t rely on the base Qwen template baked into vLLM. Point --chat-template at the file from the model repo or let vLLM auto-load the one bundled with the weights.
Tool-call parser. For agentic use (the model reports MCP-Atlas, Toolathlon, and BrowseComp scores), enable a tool-call parser compatible with the function-calling format the model was trained on, and turn on auto tool choice if your client sends tool schemas.
RoPE scaling for long context. The base context window is smaller than the advertised 262K; reaching that ceiling requires RoPE scaling (YaRN-style) configured via --rope-scaling and a matching --max-model-len. Test at intermediate lengths (32K, 128K) before pushing to the full 262K, since long-context throughput and KV cache memory both degrade non-linearly.

How do you set up SGLang instead?

SGLang follows a similar shape with its own flag names:

python -m sglang.launch_server \
  --model-path ornith-ai/Ornith-1.5-9B \
  --reasoning-parser qwen3 \
  --tool-call-parser hermes \
  --context-length 262144 \
  --json-model-override-args '{"rope_scaling": {"rope_type": "yarn", "factor": 4.0}}'

SGLang’s structured-output and constrained-decoding features are worth pairing with this model if your use case involves JSON-schema-bound tool calls, since the benchmark suite includes tool-heavy evals (ClawEval, MCP-Atlas) where malformed function-call syntax is a common source of score loss. Both servers expose OpenAI-compatible /v1/chat/completions endpoints, so client code doesn’t need to change between the two; the differences show up in throughput, scheduling behavior under concurrent long-context requests, and how quickly each project ships support for new parser formats.

What hardware do you actually need?

Remy doesn't write the code. It manages the agents who do.

AGENTS ASSIGNED TO THIS BUILD

Remy

Product Manager Agent

Leading

Design

Engineer

Deploy

Remy runs the project. The specialists do the work. You work with the PM, not the implementers.

The model card positions Ornith-1.5-9B as fitting a single 80GB GPU (A100/H100 class) in full precision at moderate context lengths. As a dense 9B model, memory scales predictably: weights take roughly 18GB in bf16, leaving the rest of an 80GB card for KV cache and activation memory. Pushing toward the full 262K context window increases KV cache substantially, so real-world deployments serving long-context workloads should budget for either a single high-memory GPU with a trimmed batch size, or quantized weights (the Mobile variant) if 80GB hardware isn’t available. If you don’t need the full context ceiling, capping --max-model-len at something like 65536 or 128000 frees up meaningful headroom for concurrent requests.

Is Ornith-1.5-9B worth running over a bigger model?

For coding and agentic tasks at the 9B scale, the published numbers make a reasonable case. It outperforms Qwen3.5-9B on SWE-bench Verified (70.6 vs 53.2), SWE-bench Pro (47.5 vs 31.3), and Terminal-Bench 2.1 (46.2 vs 21.3), and it’s within a few points of Qwen3.6-35B-A3B on several coding benchmarks despite being a much smaller dense model rather than a sparse mixture-of-experts architecture. On agentic evals it’s more mixed: it trails Qwen3.6-35B-A3B and Gemma-4-31B on MCP-Atlas and Toolathlon-Verified, suggesting tool-orchestration under long horizons still benefits from more parameters or a different scaffold.

The practical calculus: if your constraint is “one GPU, predictable latency, and strong code/reasoning performance,” Ornith-1.5-9B is a solid fit. If your workload leans heavily on multi-tool agentic chains where the larger models still lead, it’s worth benchmarking both on your own tasks before committing.

Frequently Asked Questions

What base model is Ornith-1.5-9B built on?

It extends Ornith-1.0, which itself was developed on top of Qwen3.5 and Gemma4 through continued pretraining, mid-training, and post-training, then further trained with a self-improvement reinforcement learning loop.

Does Ornith-1.5-9B support a 262K context window out of the box?

The extended context requires RoPE scaling configuration at serving time (YaRN-style scaling in vLLM or SGLang) rather than being the model’s native trained length. Test intermediate context lengths before deploying at the full ceiling.

Can I run Ornith-1.5-9B on consumer hardware?

The base model targets a single 80GB datacenter GPU. For consumer or mobile hardware, use the quantized Ornith-1.5-9B-Mobile variant mentioned in the model card, which is built for edge deployment.

Why does the chat template matter so much for this model?

The maintainers explicitly modified the standard Qwen chat template and note this was necessary “to ensure consistency between training and inference.” Using a mismatched template can silently degrade benchmark performance and break reasoning-content parsing.

Which is better for serving this model, vLLM or SGLang?

Both support the reasoning parser, tool-call parser, and RoPE scaling features this model needs, and both expose OpenAI-compatible APIs. The choice generally comes down to your existing infra, throughput needs under concurrent long-context load, and how current each project’s support is for the specific chat template and function-calling format.