Qwen 3.8 Flash Next Vision at Q4: Does Quantization Cost Accuracy?
A hands-on quad-3090 test of Qwen 3.8 Flash Next's vision support at Q4 quantization, checked against a full-precision Qwen 3.8 27B model.
What is Qwen 3.8 Flash Next and why does its vision support matter?
Qwen 3.8 Flash Next is a large multimodal model from Alibaba’s Qwen team, distributed on Hugging Face as an image-text-to-text model split across 131 safetensors shards. It recently gained working vision support in llama.cpp, merged in from an Unsloth branch, along with meaningful inference speed gains. For anyone running local vision-language models (VLMs), that combination matters: it’s rare to get both a capability update and a real throughput bump in the same merge. The catch is that most people will run this model quantized, not at full precision, and quantization has historically hit vision accuracy harder than text accuracy in local LLMs.
TL;DR
- Vision support for Qwen 3.8 Flash Next landed in llama.cpp via a merged Unsloth branch, requiring a fresh build from source to pick up both the multimodal path and recent performance work.
- Token generation speed roughly jumped from 43.99 to 62.42 tokens per second in one creator’s benchmark after the merge and follow-up updates, independent of the vision feature itself.
- A Q4 quantized version was tested against an earlier full-precision Qwen 3.8 27B run, and the quantized model was noticeably less precise on detail-heavy tasks like reading an LCD display or identifying a specific GPU model from a bare PCB photo.
- General scene description, object identification, and ingredient listing held up well at Q4, correctly identifying a false robber fly, an assassin bug (mostly), a burger cook on a griddle, and a full list of six stir-fry ingredients.
- Fine-grained tasks failed or landed close-but-wrong, including misreading a Kill A Watt display (08 read as something else) and failing to distinguish Lake Austin from Lake Travis in a landscape photo.
- Location and time-period reasoning without EXIF data was a high point, correctly narrowing an old Austin skyline photo to around 2008-2012 based on which buildings were present.
- The test ran on a quad-RTX-3090 local rig with llama.cpp, a full 262,144 token context window, mmproj set to BF16, and reasoning effort set to high, showing that even a large vision model is workable on consumer-grade multi-GPU hardware.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
How was the test set up?
The testing ran entirely locally on a rig with four RTX 3090 GPUs (a fifth card, an RTX 4090, was present in the system but not used for this run), managed through a Proxmox host and driven by llama.cpp built fresh from source. Building from source mattered here specifically because the vision merge and the associated performance improvements weren’t yet in a stable release at the time of testing.
Key configuration details from the test setup:
- mmproj (the multimodal projector component) set to BF16, even though the main model was quantized
- mmap loading enabled
- Context window set to the full 262,144 tokens
- Parallel set to 1, batch size 2048, ubatch 512
- Reasoning effort set to high, with a generous image token budget for min/max image tokens
That BF16 mmproj detail is worth noting: even in a quantized setup, keeping the vision projector at higher precision is a common practice to limit some of the accuracy loss that comes from quantizing the whole pipeline. It didn’t eliminate the gap versus full precision, but it’s the kind of tuning choice that affects real-world results more than people expect.
How did Q4 vision performance compare to full precision?
The comparison point was Qwen 3.8’s 27B variant run at full precision in an earlier test, which the creator described as producing “amazing results.” That model correctly identified a photo of a camel as being consistent with Texas Hill Country terrain, complete with shrub type and single-hump anatomy, described as nailing the call immediately.
The Q4 quantized Flash Next model, tested on the same camel photo, got the broad strokes right (dromedary, one hump, shrubs, likely Texas) but hedged toward South Texas generally rather than confidently landing on Hill Country specifically. The creator scored it as good but not quite as sharp as the full-precision run.
That pattern repeated across the test set. Tasks requiring general scene understanding, object classification, or listing multiple visible items were largely accurate:
- Identified a false robber fly correctly by name
- Identified an assassin bug correctly, though it hedged on some family-level detail
- Correctly named all six ingredients in a stir-fry photo (bell peppers, mushrooms, avocado, onions, cheese, green beans)
- Accurately described a griddle full of cooking burger patties, including melted cheese, peppers, and pan drippings, though it slightly fumbled which toppings appeared on which specific patty
Tasks requiring fine visual precision were where the quantized model struggled:
- It misread a Kill A Watt meter’s LCD display, landing close but wrong on the digit
- It failed to identify a landscape as Lake Austin, initially proposing the visually similar Lake Travis before backing off without landing on the correct answer
- Given a close-up photo of a bare GPU PCB with two 8-pin power connectors, it declined to commit to a specific model number, whereas the creator expected the full-precision 27B model would likely have identified it directly
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
The GPU PCB test is a useful illustration of the gap: enough visual detail was present (connector count, board layout) that a knowledgeable human could likely name the card, but the quantized model retreated into general, hedged language rather than making the specific call.
Is Q4 quantization worth it for local vision tasks?
It depends on the job. For general-purpose image description, object recognition, and casual visual Q&A, the Q4 version of Qwen 3.8 Flash Next performed well and was usable, especially paired with the throughput gains from the recent llama.cpp merge. Running a large vision model locally at 50+ tokens per second on consumer GPUs is a meaningful practical win regardless of some accuracy trade-offs.
For tasks that depend on reading fine text, distinguishing near-identical scenes, or identifying specific hardware or objects by exact model or type, the quantization cost showed up clearly. The gap between the Q4 run and the earlier full-precision 27B test wasn’t dramatic, but it was consistent: the full-precision model landed answers with more confidence and correctness on detail-heavy prompts.
This lines up with a pattern seen elsewhere in local VLM work: quantizing the visual pathway tends to degrade fine-detail tasks (OCR-like reading, small text, precise object identification) faster than it degrades general scene description. Anyone planning to use a quantized VLM for something like reading meter displays, transcribing screenshots, or telling apart visually similar objects should test that specific use case before relying on it, rather than assuming a good general vision demo means good performance on precision tasks.
What’s next for testing this model?
The natural follow-up, and one flagged directly in the original testing, is running the full safetensors version of Qwen 3.8 Flash Next in split mode across the same rig, without quantization, to see whether it closes the gap with the earlier 27B full-precision results. That test would be slower since it bypasses the speed benefits of quantized inference, but it would isolate whether the accuracy gap comes from quantization specifically or from other differences between the models.
Frequently Asked Questions
What is Qwen 3.8 Flash Next?
It’s a large multimodal (image-text-to-text) model from Qwen, distributed on Hugging Face across 131 safetensors shards, that recently gained vision support in llama.cpp through a merged community branch.
Do I need a special build to run vision on this model?
Yes. At the time of testing, vision support and the associated performance improvements were only available by building llama.cpp fresh from source, not from a standard release.
How much VRAM or hardware does this need?
The test ran on four RTX 3090 GPUs (24GB each) managed through a Proxmox host, with a full 262,144 token context window. Exact minimum requirements depend on quantization level and context length chosen.
Does quantization hurt vision accuracy more than text accuracy?
Based on this test, yes for fine-detail tasks. General scene description and object identification held up well at Q4, but reading an LCD display and identifying a specific GPU model from a photo both failed or landed close-but-wrong, compared to a full-precision 27B run that handled similar tasks more confidently.
What token generation speed did the update bring?
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
One benchmark cited in the test showed generation speed rising from about 43.99 to 62.42 tokens per second after the vision merge and follow-up updates, separate from the vision feature itself.
