GSQ-RCO Quantization Explained: How 1-Bit AI Models Actually Work
GSQ-RCO quantization lets labs shrink huge LLMs to near 1-bit per parameter. Here's how the bit-allocation and rounding actually work.
What is GSQ-RCO quantization?
GSQ-RCO quantization is a two-part compression technique that squeezes large language models down to roughly 1 bit per parameter while keeping most of their usefulness intact. RCO (bit-allocation) decides how many bits each part of the model deserves based on how sensitive it is to error. GSQ (rounding) then decides, weight by weight, which allowed value best represents the original number. Together they let a model that would normally need hundreds of gigabytes run on a single high-end GPU.
TL;DR
- RCO handles bit allocation, scanning the whole model and giving more bits to sensitive layers and fewer to layers that tolerate error, all while hitting a fixed total size budget.
- GSQ handles rounding, testing multiple candidate values for each weight (like -2, -1, 0, 1) and scoring them instead of just snapping to the nearest one.
- ISTA DASH Lab applied this to DeepSeek V3.1 Flash, a 763 billion parameter mixture-of-experts model, by first deleting roughly half of its experts and keeping the ones relevant to code and agent tasks.
- The “1-bit” label is an effective average, not a literal uniform encoding, since deleted experts count as zero weight and pull the average down.
- The compressed core weights come out to about 72 GB, small enough to fit on a single GPU, versus the hundreds of gigabytes a full-precision or lightly quantized version would need.
- Running the model locally consumed around 95 GB of VRAM in testing, with the option to lower context window size to reduce that further.
- Hands-on tests on coding, SQL debugging, and a full-stack bug hunt showed no hallucination loops or garbled output, which is the main failure mode labs watch for at this level of compression.
Without shadow IT.
How does RCO decide where to spend the bits?
RCO stands for the bit-allocation half of the process. The idea starts from a simple fact: not every weight in a model matters equally. Some layers are extremely sensitive to rounding error and will noticeably degrade output quality if compressed too aggressively. Others can tolerate heavy compression with barely any visible effect.
RCO starts with a fixed total size budget for the whole model, essentially a target file size. It then distributes bits across layers so that sensitive parts get more precision and tolerant parts get less, with the constraint that everything has to add up to exactly that target size. This is a global optimization problem: rather than quantizing every layer the same way, the technique spends its “bit budget” where it buys the most accuracy and saves bits everywhere else.
This is the step that happens before any individual weight gets touched. It is about deciding the shape of the compression, not the mechanics of rounding each number.
How does GSQ handle the actual rounding?
Once RCO has decided how many bits a given layer gets, each individual weight still needs to be converted into one of a small number of allowed discrete values. This is where GSQ comes in.
Standard rounding just picks whichever allowed value is numerically closest to the original weight. GSQ does something more deliberate: it evaluates several candidate values at once (for example -2, -1, 0, or 1), scores how well each one fits in context, and then iteratively trains and sharpens that choice until one option clearly wins. In effect, GSQ treats rounding as a small optimization problem per weight rather than a single snap-to-grid operation.
Put the two together and the division of labor becomes clear: RCO decides where in the model to spend precision, and GSQ decides how to best represent each number once that precision budget has been assigned. One operates at the model level, the other at the individual weight level.
Why compress a model to roughly 1 bit per parameter?
The practical reason is hardware access. A 763 billion parameter model at standard 16-bit precision would require an enormous amount of VRAM, well beyond what a single consumer or prosumer GPU can offer. Quantization exists specifically to close that gap: pushing from 16 bits down to 2 to 4 bits already shrinks a model dramatically, and pushing further toward 1 bit shrinks it again.
In the case covered here, ISTA DASH Lab took DeepSeek V3.1 Flash, a 763 billion parameter mixture-of-experts model, and first deleted roughly half of its experts, keeping the ones most relevant to coding and agentic tasks. That pruning step alone reduces the effective parameter count. Then GSQ-RCO quantization compressed what remained. The resulting core weights came out to about 72 GB, a size that fits on a single GPU rather than requiring a multi-GPU server setup.
It’s worth being precise about the “1-bit” framing. Because deleted experts contribute zero to the parameter count, they pull the average bits-per-parameter figure down. The retained, active weights are not literally stored at a uniform single bit each. The headline number is an effective average across the whole model, not a description of the format on a per-weight basis. Even accounting for that, getting a model of this scale to run with roughly a bit of information per parameter on average is a significant compression result.
What are the risks of quantizing a model this aggressively?
The most common failure mode at extreme quantization levels is degraded coherence: models that hallucinate more, get stuck in repetitive loops, or start outputting garbled tokens (including stray characters from other languages) instead of a sensible answer. This is the thing to watch for whenever a lab pushes a model below the more common 4-bit or 8-bit quantization levels.
In hands-on testing of the GSQ-RCO quantized DeepSeek coder model, none of those failure patterns showed up. The model was tested on a full-stack restaurant costing application with a deliberately planted bug (a meat-yield calculation that understated how much raw product a kitchen needed to buy), a nested SQL query with a subtle logical error (missing a status filter and averaging over the wrong grouping), and a generative coding task (rendering a rotating 3D object with lighting and surface detail). In each case the model produced coherent, usable output rather than nonsense, and in the SQL case it caught multiple real bugs and explained a judgment call it made along the way.
That said, quality differences versus a full-precision version of the same base model were visible. A 3D rendering task, for instance, came out visually flatter and less realistic compared to the non-quantized model’s output on the same prompt. That kind of gap is the expected tradeoff: extreme compression saves memory and lets the model run on far smaller hardware, at the cost of some fidelity in generation quality.
Is GSQ-RCO quantization worth using?
For anyone trying to run a very large model on a single GPU instead of a multi-GPU cluster, this kind of quantization is close to the only option. A 763 billion parameter model is simply not accessible to most individual builders or small teams at full precision. Compressing it down to a 72 GB footprint changes that calculus entirely, even with the VRAM overhead of running inference (around 95 GB of VRAM in the local test described here, adjustable downward by reducing context window size).
The tradeoff is accuracy and output quality, particularly on tasks that require more visual or creative nuance. For coding, debugging, and agentic tasks, the aggressively quantized model held up well in testing, catching real bugs and producing clean, working fixes. For tasks closer to the edge of what the model was tuned for, like generative visual tasks, the quality gap versus a full-precision model was more noticeable.
Whether it’s “worth it” depends entirely on constraints: if the alternative is not being able to run the model at all, a quantized version that performs well on your actual use case is a clear win. If GPU resources aren’t the bottleneck, a less aggressively quantized or full-precision version will generally produce higher-fidelity output.
Frequently Asked Questions
What does RCO stand for in quantization?
● Reading src/server.ts
● Editing 3 files
+ import { limit } from 'rate'
- app.use(cors())
✓ Tests passed · ready for PR
RCO refers to the bit-allocation step in this quantization approach. It decides how many bits to assign to different parts of a model based on sensitivity, giving more precision to layers that need it and less to layers that can tolerate heavier compression, all within a fixed total size budget.
What does GSQ stand for in quantization?
GSQ refers to the rounding step applied to individual weights after bit allocation has been decided. Instead of rounding each weight to the nearest allowed value, it evaluates multiple candidate values, scores them, and iteratively converges on the best fit.
Is a “1-bit” quantized model literally stored using 1 bit per weight?
Not exactly. The 1-bit figure is typically an effective average across the whole model. In the case described here, roughly half of the model’s experts were deleted entirely (contributing zero), which pulls the average bits-per-parameter number down even though the remaining active weights use more than a single bit each.
What file format is used to run these quantized models?
GGUF is the file format commonly used to store quantized model weights so they can be run with tools like llama.cpp on consumer or prosumer GPUs.
What’s the main risk when using extremely quantized models?
The main risks are hallucination, repetitive output loops, and garbled or incoherent text generation. These failure modes become more likely as bit depth drops, which is why testing aggressively quantized models on real tasks before relying on them matters.
