How to Run Nex-N2.5-mini Locally on 2x H100 GPUs
Deploy Nex-N2.5-mini with SGLang and Docker on 2x H100 GPUs, covering tensor parallelism, reasoning modes, and tool-calling setup.
What is Nex-N2.5-mini and why does hardware matter?
Nex-N2.5-mini is the smallest model in Nex-AGI’s Nex-N2.5 family, an open-weight lineup of agentic models built for tasks like computer use, web browsing, and multi-step tool execution. It’s released under Apache 2.0 on Hugging Face, using a qwen3_5_moe architecture with image-text-to-text and text-generation capabilities. Because it’s a Mixture-of-Experts model shipped across 16 safetensors shards, running it locally requires enough GPU memory and interconnect bandwidth to hold the full weight set and serve requests with reasonable throughput. Nex-AGI’s own deployment guidance specifies 2x H100 GPUs as the target configuration for this model, using tensor parallelism to split the model across both cards.
TL;DR
- Nex-N2.5-mini is the entry-level model in Nex-AGI’s three-tier N2.5 family (mini, Pro, Max), sharing the multimodal foundation of the earlier Nex-N2 but with upgrades to computer use and web-browsing agent skills.
- Official deployment guidance calls for a single node with 2x H100 GPUs running tensor parallelism at
--tp 2, a notably smaller footprint than the 8x H100 needed for Nex-N2.5-Pro or the 16x H200 multi-node setup for Nex-N2.5-Max. - Nex-AGI ships a prebuilt Docker image (
nexagi/sglang:v0.5.18-nex-patch) with a customized SGLang fork preinstalled, so there’s no need to compile a custom inference stack from source. - The model supports a configurable reasoning mode via the
reasoning_effortparameter, letting you trade latency for deeper chain-of-thought on harder tasks. - Benchmarks show mini trailing the larger Pro and Max variants by a wide margin on agentic and coding tasks (for example, 43.8 on SWE-Bench Pro versus 65.7 for Max), which matters when deciding whether the smaller footprint is worth the accuracy tradeoff.
- Tool calling is handled through the
qwen3_coderparser paired with theqwen3reasoning parser, both passed as SGLang launch flags rather than configured after the fact. - The weights are distributed as 16 safetensors shards through Hugging Face and ModelScope, with the repository also including the chat template file the launch command expects.
Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
What hardware do you actually need?
The official model card lists three deployment tiers, and they scale steeply. Nex-N2.5-Max, built on a 1.6-trillion-parameter MoE foundation, needs two nodes of 8x H200 GPUs each (16 total) with tensor parallelism of 16 and expert parallelism matched to it. Nex-N2.5-Pro runs on a single node with 8x H100 GPUs. Nex-N2.5-mini is the outlier in a good way: it’s specified to run on a single node with just 2x H100 GPUs.
That two-GPU requirement is the headline reason mini is the practical entry point for teams that want to self-host a Nex-AGI model without provisioning a multi-node H200 cluster. H100s remain expensive and scarce, but a 2-GPU node is a realistic target for a company lab or a rented cloud instance, whereas 8x H100 or 16x H200 puts the Pro and Max tiers out of reach for most self-hosters.
It’s worth noting the model card doesn’t publish a raw parameter count or per-GPU VRAM figure for mini the way it does for Max’s 1.6T total parameters. The 2x H100 recommendation should be read as the vendor’s tested, supported configuration rather than an absolute floor. Given H100s typically ship with 80GB of memory, a 2-GPU setup implies a combined memory pool in the 160GB range, split via tensor parallelism to hold the MoE weights, KV cache, and activation memory during inference.
How do you deploy Nex-N2.5-mini with Docker and SGLang?
Nex-AGI’s recommended path is a prebuilt Docker image, nexagi/sglang:v0.5.18-nex-patch, that bundles a customized fork of SGLang. This avoids the common headache of matching SGLang, CUDA, and driver versions by hand.
The documented launch command for mini looks like this:
docker run --gpus all --shm-size 32g --ipc=host \
-p 30000:30000 \
-v /path/to/your/model:/model \
nexagi/sglang:v0.5.18-nex-patch \
python3 -m sglang.launch_server \
--model-path /model \
--tp 2 \
--host 0.0.0.0 --port 30000 \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder \
--chat-template /path/to/nex-N2.5-mini/chat-template.jinja \
--mamba-scheduler-strategy extra_bufferA few details matter here. The --gpus all flag hands both H100s to the container, and --tp 2 tells SGLang to shard the model across them with tensor parallelism. The --shm-size 32g and --ipc=host flags give the container enough shared memory for multi-GPU communication, which is a common failure point if left at Docker’s defaults. The model directory gets mounted as a volume rather than baked into the image, so you download the weights separately from Hugging Face or ModelScope and point -v at that local path.
The --chat-template flag needs to reference the chat_template.jinja file that ships in the model repository alongside the safetensors shards, config.json, and tokenizer files. Skipping this or pointing at the wrong template is a common source of malformed output, since the chat template governs how the server structures system, user, and tool messages before they reach the model.
What do the reasoning and tool-calling parsers do?
Two flags in the launch command are easy to overlook but functionally important: --reasoning-parser qwen3 and --tool-call-parser qwen3_coder.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
The reasoning parser tells SGLang how to interpret and separate the model’s internal reasoning trace from its final answer in the output stream. Nex-N2.5-mini supports a reasoning_effort parameter at inference time, which controls how much the model deliberates before answering. Setting it higher pushes the model toward longer chain-of-thought for tasks that need multi-step logic, while a lower setting favors speed for straightforward requests. This is the same tradeoff pattern seen in other reasoning-tuned open models, where the parser and the effort parameter work together: the parser handles formatting, the effort setting handles behavior.
The tool-call parser, qwen3_coder, handles how the model’s function-calling syntax gets parsed into structured tool calls your application can execute. This matters directly for agentic use cases like the ones Nex-N2.5 is built for, including computer use and browser automation, where the model needs to emit calls to external functions (click here, type this, run this command) in a format your orchestration layer can reliably parse rather than free-form text you’d have to regex out yourself.
Both parsers are set as launch-time flags, not runtime API parameters, so they’re fixed for the life of the server process. If you need different parsing behavior you restart the container with different flags rather than switching per-request.
Is Nex-N2.5-mini worth self-hosting over the Pro or Max tiers?
That depends on what the workload demands. The published benchmark tables show a consistent pattern: mini scores meaningfully lower than Pro and Max across nearly every category. On SWE-Bench Pro, mini scores 43.8 versus 61.2 for Pro and 65.7 for Max. On Toolathlon Verified, mini reaches 54.6 against 68.5 for Pro and 74.7 for Max. The gap holds across coding, agentic, and multimodal benchmarks like OSWorld-Verified (71.2 for mini versus 82.2 for Pro).
The tradeoff is straightforward: mini is the only tier realistically self-hostable on commodity high-end hardware (2 GPUs versus 8 or 16), but you’re accepting a noticeable capability gap in exchange for that lower barrier to entry. For simpler agentic tasks, prototyping, or cost-sensitive deployments where 2x H100 is what you have available, mini is a reasonable starting point. For production workloads leaning on complex coding or long-horizon browser automation, the benchmark delta suggests Pro or Max (or the hosted OpenRouter endpoints Nex-AGI also offers) will perform meaningfully better if the hardware budget allows it.
Frequently Asked Questions
Does Nex-N2.5-mini require exactly 2 H100 GPUs, or is that a minimum?
Nex-AGI’s documentation lists 2x H100 as the tested single-node configuration with --tp 2 tensor parallelism. The card doesn’t publish a strict minimum VRAM figure, so 2x H100 should be treated as the vendor-supported baseline rather than a hard floor you might undercut with smaller GPUs.
Can I use a different inference engine instead of SGLang?
The official guidance is built specifically around a customized SGLang fork packaged in the nexagi/sglang:v0.5.18-nex-patch Docker image. Other engines like vLLM might work with community patches, but Nex-AGI’s documented, supported path is SGLang with this specific patched build.
What’s the difference between the reasoning parser and reasoning effort?
The --reasoning-parser qwen3 flag set at launch time controls how the server splits reasoning tokens from the final response in the output. The reasoning_effort parameter, set per request, controls how much reasoning the model actually performs before answering. One is a formatting setting, the other is a behavior setting.
Is Nex-N2.5-mini multimodal?
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
Yes. The Hugging Face model card lists it under the image-text-to-text pipeline tag in addition to text generation, and it inherits multimodal foundations from the earlier Nex-N2 model, with the N2.5 generation adding improvements to computer use and visually grounded agent tasks.
Where do I get the model weights and chat template?
Both are hosted in the same Hugging Face repository (nex-agi/Nex-N2.5-mini), mirrored on ModelScope. The repository includes 16 safetensors shards, a config.json, tokenizer files, and the chat_template.jinja file that the SGLang launch command references directly.
