Escha-W2: 2-Bit Quantization That Shrinks a 27B Model to 10GB
Escha-W2 compresses Qwen3.8-27B into 10.15GB via 2-bit quantization, matching FP8 quality while fitting 128k context on one 24GB GPU.
What is Escha-W2?
Escha-W2 is a 2-bit quantized build of Qwen3.8-27B, released by Escha Labs Inc. It packs the full 27-billion-parameter model into 10.15GB of weights, down from the multi-tens-of-gigabytes footprint a model that size normally needs. That size reduction means the whole model, its KV cache, and a meaningful context window fit on a single 24GB consumer GPU like an RTX 3090 or 4090, with enough headroom left over to push context out to 128k tokens using a tuned configuration.
TL;DR
- Escha-W2 quantizes Qwen3.8-27B to 2.469 bits per weight on average, using a mixed 2-bit/3-bit scheme per projection plus int8 for the embedding and output head, landing at 10.15GB total.
- Benchmark parity with FP8 holds up: the quantized model beats the FP8 reference on commonsense reasoning, trails by exactly one question on GPQA-Diamond, and edges ahead on LiveCodeBench within that benchmark’s noise margin.
- A hybrid attention architecture is what makes 128k context affordable: only 16 of 64 layers use full attention, and the rest are gated-delta-net layers holding a fixed ~0.15GB of recurrent state per stream that doesn’t grow with context length.
- The model ships as weights only; serving it requires a separate SGLang-based runtime (escha-runtime-qwen3dense) built with custom decode kernels that this quantization format needs.
- Reasoning effort is a tunable prompt, not a hard limit: the default
xhighsetting adds a system instruction asking the model to reason carefully, but nothing enforces a token budget unless you explicitly cap it. - Prefix caching only helps on exact repeated prompts, not on growing conversations or agent loops that append new content, because the recurrent state in the SSM layers isn’t resumable mid-sequence.
- Real-world throughput scales with VRAM and tuning: a 4090 hits 67 tokens/sec at batch size 1 with default settings, or 649 tokens/sec at 16 concurrent streams with throughput-tuned memory settings.
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
How does 2-bit quantization get near-lossless results?
Quantization normally trades accuracy for size. Cutting weight precision from 16-bit or 8-bit down to 2-bit usually degrades output quality, especially on tasks that require careful multi-step reasoning. Escha-W2 avoids the worst of that tradeoff with a mixed-precision scheme: some projections get 2-bit treatment, others get 3-bit, averaging out to 2.469 bits per weight, while the embedding and output head stay at int8. Those two layers sit at the boundary between the model’s internal representations and actual tokens, so keeping them higher precision protects the parts most sensitive to rounding error.
The result, according to the model’s own benchmark comparison against a same-backend FP8 reference, is not measurably worse across three different evaluation axes. On commonsense reasoning tasks, the 2-bit build actually scored ahead of FP8. On GPQA-Diamond, a graduate-level science question set, it fell behind by exactly one question out of the full set. On LiveCodeBench, a coding benchmark, it came out ahead, though within a margin the benchmark’s own variance can produce. None of these differences suggest a meaningful quality gap between the four-times-smaller quantized model and its full-precision counterpart.
Why does this fit on one GPU when the base model wouldn’t?
Two separate design choices stack together here. First, the 2-bit quantization itself cuts weight storage by roughly 2.9x compared to an FP8 version of the same model, per the model card’s own framing of the size reduction. Second, and just as important, is the underlying architecture: Qwen3.8-27B is a hybrid model where only 16 of its 64 layers use full attention. The other 48 layers are gated-delta-net, a linear-attention-style mechanism that holds a fixed, small amount of recurrent state (about 0.15GB per active stream) regardless of how long the context grows.
That architectural choice is what makes long context cheap. The KV cache works out to 64 KiB per token, calculated from 16 attention layers times 4 KV heads times 256 head-dimension times 2 (for keys and values) times 2 bytes per value. A conventional dense model with full attention on every layer would need four times that per token. Combined with the 2-bit weights, this is why a 24GB card can serve 128k tokens of context in a single stream, something that would be out of reach for a similarly sized dense model at any reasonable quantization level.
What hardware and software does it need?
The model card lists verified testing on three NVIDIA GPUs: the RTX 5090 (32GB, Blackwell architecture), RTX 4090 (24GB, Ada Lovelace), and RTX 3090 (24GB, Ampere). A 16GB card should technically fit the model at reduced context length, though the publisher notes this configuration is untested. The software stack requires Linux on x86-64 hardware, an NVIDIA GPU with compute capability sm_80 or newer, CUDA 12.8, and Python 3.12.
Running the model isn’t as simple as pointing a generic inference server at the weights. The quantization format needs custom decode kernels, which ship in a separate runtime repository built on SGLang, an inference serving framework. The installation process pins a specific PyTorch version (2.9.x) because the kernels are ABI-linked to it, and warns explicitly against installing a generic SGLang package from PyPI alongside the custom runtime wheel, since the two will conflict. Getting the sanity check to pass, importing torch, the custom escha kernel namespace, and sglang itself, is treated as a mandatory step before serving, because a partial check has reportedly passed on setups that couldn’t actually run inference correctly.
Is Escha-W2 worth using over a full-precision or FP8 model?
For anyone constrained to a single consumer GPU, the tradeoff looks favorable based on the published numbers. A model that would otherwise need multiple GPUs or a data center accelerator to run at 27B parameters becomes deployable on hardware many developers already own, without a measurable quality penalty on the benchmarks tested. The catch is that “measurably worse” was assessed across three specific evaluation categories (commonsense reasoning, graduate-level science questions, and live coding problems), not a comprehensive sweep of every task type. The model card itself flags that long-context retrieval quality above 64k tokens is unvalidated for this specific checkpoint, since the long-context results it draws confidence from came from a different, mixture-of-experts model architecture.
There are also real operational quirks to manage. Memory allocation settings interact in non-obvious ways: raising the context length without also raising the memory fraction allocated to the model can silently produce a token pool too small for the requested context, and with truncation enabled by default, an oversized prompt gets quietly cut rather than rejected outright. Prefix caching, which normally speeds up repeated or incrementally-extended prompts in other serving setups, only helps here on exact, complete prompt repeats. It provides no benefit to an agent loop that appends a tool result and resends, because the recurrent state in the SSM layers can’t be resumed from partway through a sequence.
How much does reasoning effort affect speed?
The model defaults to a “thinking” mode with a reasoning_effort setting of xhigh, which the documentation describes as the single biggest lever on response latency. This setting works by prepending a system instruction that asks the model to reason through the task carefully, validate assumptions, and consider alternatives before answering. A medium setting removes that instruction entirely, running the model in its default, unsteered state, while low explicitly asks it to keep reasoning brief.
It’s worth being clear about what this actually controls: it’s a prompt-level instruction, not an enforced token limit. A genuinely hard problem can still produce a long reasoning chain even at the low setting, because nothing structurally stops the model from continuing to reason. For situations where a guaranteed response time matters, such as a benchmark run or an agent loop with a timeout, the runtime supports a separate thinking budget that forces the model to close its reasoning after a fixed number of tokens, guaranteeing an answer gets produced within a bounded time.
Frequently Asked Questions
How much VRAM does Escha-W2 need to run?
The model itself takes about 18GB of VRAM at default settings on a 24GB card, including a 64k context window. Pushing to 128k context with a tuned memory fraction setting brings peak usage to roughly 22GB on the same 24GB card, based on the publisher’s measurements on an RTX 4090.
What’s the difference between Escha-W2 and a standard 4-bit or 8-bit quantized model?
Escha-W2 uses a mixed 2-bit and 3-bit scheme averaging 2.469 bits per weight, roughly 2.9 times smaller than an FP8 version of the same model. Most widely used quantization formats target 4-bit or 8-bit precision as a safer tradeoff; going to 2-bit typically risks larger quality loss, which is why the benchmark comparison against FP8 is the notable part of this release.
Can I use a generic SGLang installation to serve this model?
No. The quantization format requires custom decode kernels that ship in a dedicated runtime repository. Installing a standard SGLang package from PyPI alongside it causes conflicts, according to the model’s own setup instructions.
Does this model support multimodal input like images?
No. The model card explicitly states this checkpoint is text-only, despite the base architecture’s configuration file declaring a vision tower. The quantized weights don’t include that component.
Is long-context performance reliable above 64k tokens?
The model can technically process prompts up to roughly 128k to 147k tokens on a 24GB card, and produces coherent output at that length. However, the publisher notes that retrieval accuracy at long context hasn’t been validated for this specific model, since the long-context confidence comes from testing on a different mixture-of-experts model.
