How to Run Ornith-1.5 122B Locally with llama.cpp
A hands-on guide to serving Ornith-1.5 122B locally via llama.cpp, covering GGUF quant choices, VRAM tradeoffs, and real coding tests.
What is Ornith-1.5 122B?
Ornith-1.5 122B is an open-weight mixture-of-experts (MoE) model released by Ornith AI under the MIT license, which means it can be downloaded, run, and modified without restriction. It carries roughly 122 billion total parameters, reasons through problems before producing an answer, handles both text and images, and supports a context window above 260,000 tokens. The model card on Hugging Face lists it under the qwen3_5_moe architecture family, with weights split across 62 safetensor shards plus a standard tokenizer and processor config for multimodal input.
The pitch from Ornith AI is “stronger, faster, leaner” compared to the previous Ornith release. In practice that means shorter outputs that waste fewer tokens, tighter instruction-following, and better decomposition of multi-step tasks. It’s positioned less as a chatbot and more as a model for coding, agentic workflows, and tool use, the kind of model you’d point at a bug ticket rather than a casual conversation.
TL;DR
- Ornith-1.5 122B is an MIT-licensed MoE model with about 122 billion parameters, reasoning capability, image input support, and a context window over 260,000 tokens.
- It runs locally through llama.cpp using GGUF quantized weights, with VRAM use scaling heavily based on the chosen context length rather than just model size.
- A hands-on bug-fix test on a legacy law-firm inheritance app had the model locate and correct a one-line logic error without step-by-step guidance from the user.
- On benchmark comparisons, Ornith-1.5 reportedly outperforms similarly sized models like GLM flash-next and Nvidia’s Nemotron 3.5 Super on coding and agentic tests, and competes with models two to four times its size on SWE-Bench Pro style tasks.
- The model handled a vision recipe test, a physics/circuit reasoning puzzle, and a single-file HTML animation task convincingly, though its reasoning traces ran long even on simple image questions.
- Its multilingual output across roughly 80 languages was solid for major languages (Spanish, French, German, Japanese, Korean, Arabic) but weaker and less reliable for low-resource languages like Thai, Zulu, and Hawaiian.
- Choosing the right GGUF quant is the main lever for fitting this model on consumer or prosumer GPUs, trading precision for lower VRAM footprint.
How do you set up Ornith-1.5 122B with llama.cpp?
Running Ornith-1.5 locally follows the same general path as other large open-weight releases: pull a GGUF-quantized version of the model, point llama.cpp’s server at it, and configure context length and GPU offload to match your hardware.
A GGUF build of Ornith-1.5 (referenced in testing from a community quantizer going by “Atomic Chat”) made it possible to serve the model through llama.cpp’s server mode with a single launch command. The server exposes both an API and a basic web GUI, which is enough to run text and image prompts directly without extra tooling. Because the model accepts image input, you also need the matching processor and preprocessor configs bundled with the weights (these ship alongside the safetensors files on the model’s repository) so llama.cpp can route image tokens correctly.
The practical workflow looks like this:
- Download a GGUF quant that matches your available VRAM.
- Launch
llama.cpp’s server with the model path, desired context length, and GPU layer offload settings. - Connect through the built-in web UI or point an external agent framework at the local API endpoint.
- Monitor VRAM usage in real time and adjust context length if you hit memory limits.
How much VRAM does Ornith-1.5 122B need?
VRAM consumption depends far more on your context length setting than on the base model size alone. In testing, VRAM usage ran high specifically because a large context window was configured, not because the quantized weights themselves demanded it. Reducing the context length is the simplest way to bring memory use down if your GPU can’t handle the full 260,000+ token window at once.
This is the core tradeoff anyone running Ornith-1.5 locally has to manage: a MoE model this size, even quantized, is already a meaningful VRAM commitment, and stretching the context window toward its full rated length multiplies that cost further. For most local setups, picking a mid-range GGUF quant and a trimmed context length (well below the maximum) is the realistic starting point, scaling up only if your hardware allows it.
GGUF quantization is what makes a 122B-parameter MoE model runnable outside a data center at all. Lower-bit quants shrink the model’s footprint substantially at some cost to output precision, while higher-bit quants preserve more of the original model’s behavior but demand more VRAM. The right choice depends entirely on what GPU (or combination of GPUs) you’re working with, and testing across multiple quant levels is the only reliable way to find your personal sweet spot.
Is Ornith-1.5 122B good at coding and agentic tasks?
Based on hands-on testing, yes, it performs well on real debugging work. In one test, the model was given access to a full-stack legacy application (a law-firm inheritance calculator running in Docker) with a genuine logic bug: a beneficiary who died before the person who died was still receiving a payout, when the will required surviving by 30 days to qualify. The model was handed only a goal, with no hints about where the bug lived.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
It correctly identified the flawed line, fixed it, and flagged some related false positives along the way, without breaking the rest of the application’s ledger logic. That kind of autonomous, scoped bug fix is exactly the agentic/coding use case Ornith AI says the model is built for, and the test held up.
On benchmark charts, Ornith-1.5 reportedly beats models of similar size, including GLM’s flash-next variant and Nvidia’s Nemotron 3.5 Super, on most coding and agentic benchmarks. On SWE-Bench Pro, which uses real-world software issues rather than synthetic ones, it reportedly leads the comparison set shown. It also reportedly performs competitively against DeepSeek’s flash variant and against much larger models, despite being a fraction of their size, though it doesn’t win across every single benchmark category.
How does it handle vision and reasoning tasks?
The model handles image inputs by reasoning through what it sees before answering, which showed up clearly across several test prompts. Given a photo of a curry dish, it weighed multiple regional possibilities (Thai, Caribbean, North Indian) before settling on an identification, then produced a full recipe with ingredient quantities and numbered steps. Given an image-based physics puzzle about which people in a chain would receive an electric shock, it correctly identified the people with a complete circuit path and explained why, including a summary table of its reasoning.
The tradeoff is latency. Its thinking traces ran long even on fairly simple image questions, going back and forth between candidate answers before committing. That thoroughness produced accurate results in testing but means the model isn’t necessarily “lean” in its default reasoning depth. Reasoning effort can generally be constrained through prompt or config settings if faster, shorter answers matter more than exhaustive deliberation.
On a creative coding test, asked to generate a single self-contained HTML file animating a tattoo being drawn, the model produced a working animation with outline, shading, and color fill stages, with no external libraries or dependencies. It’s not an image generation model, so the artistic quality has clear limits, but as a demonstration of procedural SVG/canvas-style coding in one shot, it held up well.
Frequently Asked Questions
What license is Ornith-1.5 122B released under?
It’s released under the MIT license, meaning the weights can be downloaded, run, and built upon without the restrictions found in more limited “open” licenses.
What architecture does Ornith-1.5 122B use?
It’s a mixture-of-experts (MoE) model built on a qwen3_5_moe architecture base, with around 122 billion total parameters and multimodal (text and image) input support.
Can Ornith-1.5 122B run on a single consumer GPU?
It depends on the GGUF quant and context length chosen. Lower-bit quants and a reduced context window bring VRAM requirements down significantly, but running near the model’s full 260,000+ token context will require substantial VRAM regardless of quant level.
How does Ornith-1.5 122B compare to its predecessor?
Testing described it as a clear improvement over the earlier Ornith model, particularly in vision understanding and response quality, though exact version-to-version benchmark numbers for the prior release weren’t detailed.
Is Ornith-1.5 122B good at non-English languages?
It handled major world languages (Spanish, French, German, Japanese, Korean, Arabic, Hebrew) reasonably well in a multilingual translation test, but struggled with low-resource languages like Thai, Zulu, and Hawaiian, producing output that looked inaccurate or invented.
