Ternary Bonsai 2 9B Tested: How Much Does 2-Bit Quantization Cost You?
Hands-on tests of the 2-bit ternary Bonsai 2 9B model across coding, multilingual, reasoning, and vision tasks reveal where quality actually breaks down.
What is Ternary Bonsai 2 9B?
Ternary Bonsai 2 9B is a heavily compressed version of a 9 billion parameter vision-language model, built by PrismaML on top of Qwen 3.5 9B, that stores most of its weights using only three possible values: -1, 0, or +1. Instead of the usual 16-bit floating point numbers, each weight gets squeezed into a ternary state, with a small scaling factor kept per group of 128 weights to preserve rough magnitude. The result is a model that shrinks from roughly 17.9 GB down to about 2.4 GB in GGUF format, a reduction of roughly seven times, while keeping the original’s 262K token context window and image-understanding capability.
TL;DR
- Ternary quantization compresses weights to just three values (-1, 0, +1) instead of 16-bit floats, cutting Ternary Bonsai 2 9B down to about 2.4 GB from an original 17.9 GB Qwen 3.5 9B base.
- The model runs locally through llama.cpp, released under Apache 2.0, and keeps the full 262K context window, though VRAM use scales with however much of that context you actually load.
- On a coding and vision task (animating a donair kebab from a reference photo), the model produced working code after a reasoning pass, suggesting the compression didn’t wreck core generation ability.
- Multilingual output degrades with language rarity: major languages like Spanish, Portuguese, and Mandarin came out strong, mid-resource languages like Hindi and Greek got garbled, and rare languages collapsed into repeated nonsense.
- In a scientific reasoning test involving a wind tunnel infographic, the model correctly flagged a bogus, AI-generated report as non-credible, but invented numbers and made arithmetic errors along the way.
- Vision description stayed strong even when creative writing fell apart: the model nailed a complex scene description but lost plot logic and consistency in a follow-up storytelling task.
- Compared to earlier Bonsai models, this one shows a real quality loss relative to the full-precision Qwen base, but represents a meaningful step up from previous ternary attempts in the same family.
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
How does ternary quantization actually shrink a model?
A standard large language model stores each weight as a 16-bit floating point number, something like 0.82 or -0.07. That precision adds up fast across billions of parameters, which is why a 9 billion parameter model in full precision weighs in at nearly 18 GB.
Ternary quantization throws almost all of that precision away. Instead of a wide range of possible values, each weight gets mapped to one of three states: -1, 0, or +1. To avoid losing all sense of relative scale, the model keeps one small scaling number for every group of 128 weights, which helps reconstruct something close to the original weight’s magnitude during inference. Fewer bits per weight means a dramatically smaller file: in this case, roughly 17.9 GB down to 2.4 GB, close to a sevenfold reduction.
The tradeoff is obvious: you’re throwing away granularity in exchange for size. The open question, and the one worth testing, is how much actual capability survives that compression.
How well does it handle coding and vision tasks?
In a hands-on test, Ternary Bonsai 2 9B was given a real photograph of a donair kebab and asked to generate code for an animated version of it spinning in front of a flame, rendered live in-browser. The model reasoned through the prompt first, then produced code that successfully built the animation based on what it saw in the photo.
This is a meaningful result for a 2.4 GB model. Tasks that combine image understanding with code generation require the model to extract visual details accurately and then translate them into working logic, two different competencies that often degrade independently under heavy quantization. The fact that both held up in this case suggests the compression process didn’t gut the model’s core reasoning and generation pathways, at least for a task like this.
Does it still reason well at this size?
Reasoning quality was tested with a deliberately tricky scientific task: a busy wind tunnel infographic containing a mix of real and fabricated data, including conflicting Reynolds numbers, garbled text, and nonsense formulas. The model was asked to review it the way a senior aerodynamicist would.
It performed well at the macro level. It flagged the conflicting Reynolds number, called out the garbled text and bogus formulas, and correctly concluded the report wasn’t credible, a reasonable judgment given the image was intentionally stuffed with AI-generated nonsense.
Where it broke down was in the details. The model misread small numbers on the chart, invented a stall angle that didn’t actually appear anywhere in the image, and made an arithmetic slip during its own physics check (confusing units, among other errors). The takeaway: the model can reach a sensible high-level conclusion about a complex, flawed document, but you can’t trust the specific numbers it cites along the way. Hallucination is still present, though reportedly improved compared to earlier models in the same ternary family.
How does it perform across languages?
One coffee. One working app.
You bring the idea. Remy manages the project.
Multilingual testing involved asking the model to describe a single restaurant scene in 28 languages spanning multiple continents. The pattern that emerged was consistent: quality tracked closely with how much training data a language likely has.
High-resource languages like Spanish, Portuguese, and Mandarin came through strong. Korean and Indonesian showed small slips. Mid-resource languages, including Hindi, Urdu, Greek, and several African languages, came out garbled, with stray words from Russian and Chinese leaking into the output. Rarer languages fared worst, collapsing into repeated, nonsensical text rather than coherent sentences.
This is a predictable failure mode for a heavily compressed model asked to stretch across languages it saw less of during training, and it’s worth noting the test intentionally pushed into territory where even the full-precision base model would likely struggle. Still, the gradient is clear: the rarer the language, the more the ternary compression shows its seams.
Can it handle creative and cultural writing?
A more unusual test combined vision description with cultural storytelling: the model was shown an AI-generated image of a man with a stained shirt caught between two rival mother-in-law figures, one Italian, one Pakistani, each pushing their daughter as a match. The model was asked to describe the scene, then write a funny sketch mixing Italian and Urdu phrases with a twist ending.
The image description was accurate, correctly identifying every person, dish, and detail including the stain. The Italian dialogue worked reasonably well, including at least one genuinely funny line. But the actual narrative fell apart: the Urdu dialogue was weak and repetitive, the central premise (the daughters and marriage angle) never got mentioned, a clothing detail changed inconsistently partway through, and the twist ending didn’t logically follow from the setup.
The conclusion here mirrors the reasoning test: strong perception, weak execution on anything requiring sustained logical or narrative consistency. Vision grounding and language mimicry survived quantization better than multi-step creative planning did.
Is Ternary Bonsai 2 9B worth using?
For anyone who wants a vision-capable, 262K-context local model running locally in roughly 2.4 GB of VRAM, this model clears a meaningful bar: it generates working code from images, reasons sensibly about flawed data, and describes complex scenes accurately. That’s a lot of capability packed into a file size seven times smaller than its base model.
But the limitations are real. Numerical accuracy degrades, rare-language output breaks down, and multi-step creative or logical consistency (plots, character details, follow-through) suffers noticeably. Compared to the full Qwen 3.5 9B base, there’s a clear capability gap. Compared to earlier Bonsai ternary models from the same team, this release represents a tangible improvement, even if it still shows the hit-or-miss behavior the format is known for.
Frequently Asked Questions
What makes a model “ternary”?
A ternary model restricts most of its weights to just three possible values (-1, 0, or +1) instead of the wide range allowed by standard 16-bit floating point numbers. A small per-group scaling factor helps approximate the original weight magnitudes, enabling drastic file size reduction at some cost to precision.
How much smaller is Ternary Bonsai 2 9B than its base model?
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
The original Qwen 3.5 9B model weighs about 17.9 GB. The ternary quantized version in GGUF format comes in at approximately 2.4 GB, close to a sevenfold reduction in size.
Can Ternary Bonsai 2 9B run on consumer hardware?
Yes, it’s designed to run locally through llama.cpp. VRAM consumption depends heavily on context length; the full 262K token context window demands more memory, but reducing the context parameter brings requirements down for smaller GPUs.
Where does the model lose the most quality compared to its full-precision base?
The biggest drops show up in rare-language output, precise numerical reasoning, and multi-step creative or logical consistency. Image description, code generation, and high-level reasoning judgments held up comparatively well.
Is Ternary Bonsai 2 9B free to use?
Yes. The weights are released under the Apache 2.0 license, which allows free use, modification, and redistribution.
