Spark X2.5 4B: How Does This Small Model Run Locally?
Hands-on test of Spark X2.5 4B locally via Docker and SGLang, covering VRAM use, a 1M context claim, coding tasks, and multilingual gaps.
What is Spark X2.5 4B?
Spark X2.5 4B is a 4 billion parameter open weight language model released by XHToken under an Apache 2.0 license, with weights hosted on Hugging Face. Its pitch is doing more with less: a hybrid attention design that combines lightweight sliding window attention layers with occasional full attention layers, letting a model this small claim a native 1 million token context window, a number usually reserved for models many times its size. The family also includes a smaller 1.7B variant. The model card lists training on roughly 20 trillion tokens, support for over 200 languages, and native compatibility with agentic tool-calling harnesses like Hermes.
TL;DR
- Spark X2.5 4B uses a hybrid attention pattern, three sliding window layers for every one full attention layer, which is how it reaches a claimed 1 million token context without the usual compute cost.
- On a 48GB GPU, the model itself loaded at under 9GB of VRAM, and serving it with a KV cache under SGLang pushed usage to around 41GB at a 65k context cap.
- In an agentic bug-fix test on a live four-tier dispatch dashboard app, the model found and corrected a triage-ordering bug correctly, but took three to four times longer than larger models to get there.
- A single-file HTML/CSS animation test (a rotating kebab with flame and grease effects) produced a recognizable but visually flat result, with the model burning 30 to 40 minutes of reasoning tokens and getting stuck in loops before recovering.
- The multilingual claim breaks down outside major languages: European languages translated cleanly, but Indic, Southeast Asian, and African language outputs were often garbled, mixed with English filler, or contaminated with Chinese characters, and one language collapsed into a row of equals signs.
- The model is served through a Docker image for SGLang, exposing an OpenAI-compatible API, which makes it straightforward to plug into existing tooling that expects a standard chat completions endpoint.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
How does the hybrid attention architecture work?
The core trick behind Spark X2.5’s long context claim is architectural, not just data volume. Standard transformer models use full (global) attention at every layer, where each token can attend to every other token in the sequence. That gets expensive fast as context grows, since compute and memory scale with the square of sequence length.
Spark X2.5 instead repeats a block made of three sliding window attention layers followed by one full attention layer. The sliding window layers only look at a small nearby chunk of tokens, which is cheap and fast. The single global layer per block still gives the model a way to connect information across the entire sequence, just far less often than in a conventional design. Stacked repeatedly from embedding to output, this pattern is what lets a 4 billion parameter model advertise a native 1 million token window, a spec normally seen on much larger, more expensive models.
How much VRAM does Spark X2.5 4B actually need?
In a hands-on test using an Ubuntu machine with a single 48GB GPU, the raw model weights loaded at just under 9GB of VRAM. Once served through SGLang with an active KV cache, VRAM consumption rose to roughly 41GB at a context length capped at 65k tokens (a deliberate reduction from the full 1M window to fit comfortably in available memory).
That gap between “weights loaded” and “weights plus KV cache” is the practical number to plan around. A 4B model is genuinely light on disk and at rest, but pushing toward its long-context ceiling will demand meaningfully more memory, and the 41GB figure was already at a fraction of the model’s theoretical max context. Runners with less VRAM should expect to cap the context window lower still, which is a configurable flag when serving the model.
How is it served, and does the setup match its “lightweight” positioning?
The model was served using a pinned Docker image for SGLang, an inference engine that spins up an OpenAI-compatible API server, in this case exposed on port 30000. The launch command sets flags telling the server how to parse the model’s tool-calling format (Spark’s own template) and its reasoning output style (compatible with Qwen 3 conventions), along with the context length cap.
This is a fairly standard modern local-serving workflow: pull the weights, pull a pinned inference image, pass a handful of flags, and get an API endpoint that drops into anything expecting OpenAI-style chat completions. For anyone already running local models through SGLang or similar servers, Spark X2.5 doesn’t require unusual tooling.
Does it hold up on real agentic coding tasks?
The most demanding test involved a live four-tier web application (Postgres database, FastAPI backend, Nginx frontend, all orchestrated with Docker Compose) simulating an emergency dispatch dashboard. The app had a deliberately planted bug: within the same priority band, incidents were sorted by fewest people affected instead of most, meaning a large-scale emergency could rank behind a minor one.
Remy is new. The platform isn't.
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
Using the Hermes agentic harness with no hints or hand-holding, Spark X2.5 4B was asked to find and fix the bug on its own. After an extended reasoning process, roughly 16 to 17 minutes, it correctly identified and repaired the sorting logic, verified by a reset of the dashboard showing the correct incident at the top. The result matched what larger models produce on the same test, but the model took three to four times longer to reach the correct fix.
A second test asked the model to build a rotating kebab animation in a single self-contained HTML file, no libraries, testing procedural graphics: rotation, flame flicker, heat haze, grease drips. The model spent 30 to 40 minutes reasoning, visibly looping and recovering multiple times before finishing. The output captured the general shape and warm color banding of the concept but fell short on detail: the meat looked overly symmetrical and thin, the flame element read as a flat glowing rectangle rather than something dynamic, and the overall scene looked more like a static 2D banner than a live animation. Called an average result, on par with or slightly below what larger competing models produce on the same prompt, but reasonable for a 4B parameter model.
Is the multilingual claim reliable?
This is where the model showed its clearest weakness. Asked to translate a single sentence into a wide spread of the 200-plus languages it claims to support, performance split sharply by language family. Major European languages (Spanish, French, Romanian, Nordic languages) translated cleanly, which isn’t a high bar since most models handle these well.
Outside that zone, quality dropped fast. Several Indic, Southeast Asian, and African language outputs were garbled, mixed with stray English filler words, or bled Chinese characters into unrelated target languages like Punjabi. At least one language (Tigrinya) failed outright, returning a row of equals signs instead of any translation. The practical read: the 200-plus language claim is attempted, but reliability is concentrated in well-resourced, high-data languages. Low-resource languages are where a 4B parameter budget clearly runs out of room.
Is Spark X2.5 4B worth running locally?
For its size, Spark X2.5 4B performs above what a 4 billion parameter count usually suggests, particularly on agentic coding and bug-fixing tasks where it reached correct answers that matched larger models, just slower. Its VRAM footprint at rest is genuinely small, and the hybrid attention design is a legitimate approach to squeezing long context out of a compact model. Where it falls short is in reasoning speed (extended thinking loops that can run 15 to 40 minutes on nontrivial tasks) and in the long tail of its multilingual claims, where low-resource languages produce unreliable or broken output. It’s a reasonable pick for local agentic experimentation on modest hardware, less so if broad multilingual accuracy is a requirement.
Frequently Asked Questions
What is the context window of Spark X2.5 4B?
The model card claims a native 1 million token context window, achieved through a hybrid attention design using sliding window layers combined with periodic full attention layers rather than full attention throughout.
How much VRAM do you need to run Spark X2.5 4B?
Bare weights loaded at under 9GB of VRAM in testing. Serving the model with an active KV cache at a 65k token context cap pushed usage to around 41GB on a 48GB GPU. Actual requirements will scale with how much context length is enabled.
Can Spark X2.5 4B handle agentic coding tasks?
Yes, in testing it correctly diagnosed and fixed a sorting bug in a live four-tier web application using an autonomous agentic harness with no hints, though it took several times longer than larger models to reach the same correct answer.
Is the 200+ language support claim accurate?
Only partially. Major European languages translate reliably, but many Indic, Southeast Asian, and African languages produced garbled text, English filler injection, or cross-language character bleeding in testing, with at least one language failing entirely.
What tools are used to serve Spark X2.5 4B locally?
It was served using a pinned Docker image for SGLang, which exposes an OpenAI-compatible API endpoint, making it compatible with existing tools and harnesses built for standard chat completion APIs.