NVIDIA Nemotron 3.5 Lightning: A 30B MoE Built for Agent Grunt Work
NVIDIA's Nemotron 3.5 Lightning is a 30B-A3B open MoE model built for fast, cheap agent execution. Here's what its architecture and benchmarks mean.
Drafted with Claude from source material, checked by automated verification, and reviewed before release. How we make these.
What is NVIDIA Nemotron 3.5 Lightning?
Nemotron 3.5 Lightning is an open-weight mixture-of-experts language model from NVIDIA with 30 billion total parameters and roughly 3 billion active per token (30B-A3B). It’s distilled from the larger Nemotron 3 Ultra model and built specifically for the “execution layer” of agentic systems: tool calls, retrieval, validation, formatting, classification, and summarization. It isn’t designed to be a frontier reasoning model. It’s designed to be fast, cheap, and fine-tunable for narrow, repetitive work that agents do constantly.
TL;DR
- Nemotron 3.5 Lightning is a 30B total parameter, 3B active parameter MoE model NVIDIA released as open weights, including post-training data and recipes.
- It targets the agent execution layer, the boring high-volume work like tool calls, retries, formatting, and validation that eats most of an agent’s token budget.
- NVIDIA claims roughly 4x throughput versus similarly-sized Qwen models and around 30-35% faster performance than Qwen 3.6 MoE on a 10,000-task agentic benchmark called Pinch Bench.
- The speed comes from a hybrid Mamba-transformer architecture, a built-in multi-token predictor, and speculative decoding support via DeepSeek-derived methods called D-Flash and a DGX Spark-tuned variant called D-Spark.
- On the Artificial Analysis intelligence index, it lands roughly on par with GPT-OSS but runs at meaningfully higher throughput.
- It ships as both a BFloat16 checkpoint and an NVFP4 quantized version tuned for NVIDIA hardware, and companies like CrowdStrike, CodeRabbit, and Base10 have already fine-tuned it for production use.
- It pairs naturally with SwitchYard, NVIDIA’s open-source model router, letting teams send simple tasks to Lightning locally and escalate hard ones to a frontier model over an API.
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
Why does an agent need a “grunt work” model at all?
Most of what an autonomous agent does isn’t hard. Look at the trace of a long-running coding or ops agent and a huge share of the steps are mechanical: call a tool, check the output format, retry on a transient error, summarize a log, classify an alert, decide whether to use a tool at all. None of that needs frontier-level reasoning. It needs a model that responds fast, follows instructions reliably, and doesn’t cost much per call.
The problem is that most teams route all of that work through the same expensive, general-purpose model they use for the hard reasoning steps, because that’s the model they already have API access to. NVIDIA’s pitch with Nemotron 3.5 Lightning is that this is wasteful. If 90% of an agent’s tokens go toward tasks a small specialized model can handle just as well, running everything through a frontier model is burning money and adding latency for no benefit.
How does Nemotron 3.5 Lightning get its speed?
Three architectural choices drive the throughput claims.
First, it uses a hybrid Mamba-transformer design, consistent with the rest of the Nemotron 3 family. This lets it hold onto a very large context window (NVIDIA cites up to a million tokens) far more cheaply than a pure transformer would, since Mamba-style state space layers scale better with sequence length than standard attention.
Second, it’s a “latent MoE.” Incoming tokens get projected into a smaller latent space before being routed to one of the model’s experts, which NVIDIA says improves accuracy per byte of activated compute. It has 30 experts total but only activates a small subset per token, which is what keeps the active parameter count down at roughly 3B despite the 30B total size.
Third, and probably most important for real-world speed, is speculative decoding baked in from the start. The model includes a multi-token predictor (MTP) trained in during continued pretraining, plus support for D-Flash, a speculative decoding drafting technique derived from DeepSeek’s published methodology, and D-Spark, a variant specifically tuned for NVIDIA’s DGX Spark hardware. In hands-on testing on a DGX Spark, one reviewer measured around 71 tokens per second on a single stream, with the multi-token predictor’s guesses accepted about 70% of the time, a strong hit rate that translates directly into fewer sequential decoding steps and faster wall-clock generation.
How fast is it compared to Qwen?
NVIDIA’s own comparisons position Lightning against similarly-sized models in the Qwen family, claiming roughly 4x the throughput. On Pinch Bench, an agentic benchmark NVIDIA ran across 10,000 tasks, Lightning came in 30-35% faster than Qwen 3.6 MoE. On general intelligence, Lightning tracks close to GPT-OSS on the Artificial Analysis index, meaning it’s not claiming to be smarter than comparable open models. It’s claiming to do similar work markedly faster.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
That distinction matters. In one head-to-head test cited by a reviewer, a 1,400-token reasoning task ran in about 20 seconds on Lightning locally on a DGX Spark, versus about 50 seconds for Kimi K3 called over an API. Lightning wasn’t more capable on that task. It was faster, and it ran entirely on local hardware, which also matters for anything involving sensitive data.
Is Nemotron 3.5 Lightning worth using instead of a frontier model?
It depends entirely on the task. This is not a coding agent’s brain, and it’s not meant to be. Testing shown by one reviewer found it handles agentic basics well: retrying after injected errors, chaining multiple tool calls, knowing when not to call a tool at all, and holding up over long-running tasks. Where it struggles is resisting prompt injection and anything requiring extended chains of thought, since it doesn’t produce much in the way of reasoning tokens.
The practical implication is that Lightning works best as a specialist sitting underneath an orchestrator, not as the orchestrator itself. You’d use it for the repetitive execution steps of an agent loop and keep a stronger model (or a human) responsible for judgment calls, security-sensitive decisions, and anything where getting it wrong is expensive.
NVIDIA is leaning into this positioning by pairing Lightning’s release with SwitchYard, its open-source routing layer. SwitchYard offers four routing strategies: random split for A/B testing, an LLM classifier that reads a request and picks a tier, a “stage router” that reads existing signals like failing tool calls to decide the path, and an escalation mode where every task starts cheap and only gets promoted to a bigger model when a judge model flags real trouble. Combining a router like this with Lightning as the cheap tier and a frontier model as the escalation tier is the core use case NVIDIA is pitching: keep routing overhead low (the stage router approach costs nothing extra since its signals are already in the conversation) while avoiding the cost of running every single step through an expensive model.
What are companies actually doing with it?
The more interesting story than the base model itself is what NVIDIA’s customers are doing with it. Because Lightning ships with full post-training data and recipes, not just weights, teams have been fine-tuning it cheaply for narrow use cases. NVIDIA’s blog cites CrowdStrike reaching Nemotron 3 Super level accuracy on their specific task at roughly a fifth of the cost by fine-tuning Lightning. CodeRabbit and Base10 reportedly fine-tuned it for one epoch in under three hours for about $100 and used the result as a routing component in a larger system. Unsloth has published scripts for fine-tuning it on consumer hardware, which puts this kind of customization within reach of small teams, not just enterprises with dedicated ML infrastructure.
The model is released under NVIDIA’s Open Model license, which permits commercial use and distillation of outputs without attribution requirements. It’s text-in, text-out only. There’s no vision support, NVIDIA has that covered separately in its Nano Omni model line.
Frequently Asked Questions
What does A3B mean in Nemotron 3.5 Lightning’s name?
It refers to the model’s mixture-of-experts structure: 30B total parameters with about 3B active per token, since MoE models only run a subset of their experts for any given input.
Can I run Nemotron 3.5 Lightning locally?
Remy is new. The platform isn't.
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
Yes. NVIDIA ships it in both BFloat16 and NVFP4 quantized formats, with the NVFP4 checkpoint tuned for NVIDIA hardware including consumer RTX cards and the DGX Spark.
Does Nemotron 3.5 Lightning replace a coding agent’s main model?
No. It’s built for the execution layer of agent workflows, things like tool calls, retries, and formatting, not for the reasoning-heavy work of a primary coding or planning model.
How does it compare to Qwen models?
NVIDIA claims roughly 4x the throughput of similarly-sized Qwen models and 30-35% faster performance than Qwen 3.6 MoE on its Pinch Bench agentic benchmark, while landing near GPT-OSS on general intelligence measures.
What is SwitchYard and how does it relate to this model?
SwitchYard is NVIDIA’s open-source model routing layer, released alongside Lightning, that decides whether a task should go to a fast local model like Lightning or escalate to a more capable frontier model based on cost, speed, and task complexity.
