Recursive Flint's SWE-bench and Terminal-Bench Scores, Explained
Recursive Flint hits 89.7% on SWE-bench Pro v2 and 54.9% on Terminal-Bench 2.1. Here's what those numbers mean for a 40B open coding model.
What are Recursive Flint’s benchmark scores?
Recursive Flint, an open-weight mixture-of-experts model built for coding agents, scores 89.7% on SWE-bench Pro v2, 74.2% on SWE-bench Verified, and 54.9% on Terminal-Bench 2.1, according to its own model card. These numbers come from Recursive’s release evaluations, run with Terminus 2 for Terminal-Bench and mini-swe-agent for SWE-bench, using temperature 1.0, top-p 0.95, and top-k 20. The model has 39.8 billion total parameters with roughly 2.5 billion active per token, placing it in a size class meant to run on a single high-end GPU rather than a datacenter cluster.
TL;DR
- Flint reports three headline scores: 89.7% on SWE-bench Pro v2, 74.2% on SWE-bench Verified, and 54.9% on Terminal-Bench 2.1, all from Recursive’s own evaluation harness.
- The model is a sparse mixture-of-experts design: 39.8B total parameters but only about 2.5B active per token (including the output head), which is why Recursive markets it as a “40B-A2.5B” model.
- Evaluations used a fixed agent stack: mini-swe-agent for the SWE-bench suites and Terminus 2 for Terminal-Bench, with agentic scores averaged over at least five pass@1 attempts.
- Flint penalizes web lookups on Terminal-Bench: its reported score counts any solution obtained through web retrieval as a failure, a stricter rule than some developer-reported comparison scores apply.
- Hardware requirements are modest for a model this capable: Recursive says one B300 or one H200 GPU per replica is enough to serve the BF16 checkpoint via its SGLang kit, with quantized builds available separately for NVIDIA and Apple Silicon.
- The benchmark comparisons target “local models”: Recursive’s own benchmark chart frames Flint against other models sized to run on similar single-GPU hardware, not against frontier closed models.
How were these benchmark numbers actually measured?
Recursive ran Flint’s agentic evaluations with two established harnesses rather than a custom one-off script. SWE-bench Pro v2 and SWE-bench Verified were scored using mini-swe-agent, a lightweight agent scaffold built for the SWE-bench family of tasks, where a model has to read a GitHub issue, explore a real codebase, and produce a patch that passes hidden tests. Terminal-Bench 2.1 was scored using Terminus 2, an agent harness for the Terminal-Bench suite, which tests whether a model can complete multi-step command-line tasks inside a sandboxed terminal.
Sampling settings were fixed across the board: temperature 1.0, top-p 0.95, top-k 20. Because agentic tasks have some run-to-run variance, Recursive reports pass@1 averaged over at least five attempts per task rather than a single run. The model card also flags which numbers were measured directly by Recursive (marked with a dagger in its benchmark chart) versus which comparison figures came from other developers’ own reported scores, a distinction worth keeping in mind since evaluation setups are not always identical across labs.
Why does the Terminal-Bench score come with a caveat?
One detail in Flint’s evaluation writeup stands out: Recursive explicitly counts any Terminal-Bench solution obtained through web retrieval as unsuccessful. In other words, if the agent solved a task by searching the web for an answer rather than reasoning through the terminal environment itself, that attempt is scored as a failure, even if the task technically got done.
This matters because not every developer-reported benchmark applies the same rule. A model that’s allowed to use web search during Terminal-Bench evaluation could post a higher score for reasons that have nothing to do with its actual command-line reasoning ability. By disallowing that shortcut, Flint’s 54.9% on Terminal-Bench 2.1 is arguably a stricter, more conservative number than it might otherwise be, which is a useful thing to know before lining it up against another model’s self-reported score on the same benchmark.
How does Flint’s architecture relate to its scores?
Flint is a decoder-only mixture-of-experts transformer with 49 layers: two dense layers followed by 47 MoE layers. Each MoE layer has 256 routed experts, of which 6 are selected per token, plus a shared expert that’s always active. That sparse routing is how the model reaches 39.8 billion total parameters while only activating about 2.5 billion per token (2.2 billion in the active body alone, with the rest coming from the output head).
The attention stack mixes 35 sliding-window layers (a window of 513 tokens, meaning the current token plus the 512 before it) with 14 full-attention layers. Sliding-window layers use RoPE for position encoding; the full-attention layers use none. The documented context configuration is 262,144 tokens covering both prompt and output. This architecture choice, heavy sparsity plus a mix of local and global attention, is part of why Recursive can claim strong agentic coding scores while keeping inference cheap enough to run on a single GPU.
Is Flint’s size-to-performance ratio actually competitive?
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
The honest answer is that Flint’s model card frames its benchmark chart specifically as a comparison against other “local models” that run on similar hardware, not against the largest frontier models from major labs. Recursive’s own benchmark visualization compares Flint’s BF16 and NVFP4 checkpoints against other models sized for single-GPU deployment. That’s a meaningful distinction: a 74.2% on SWE-bench Verified and an 89.7% on SWE-bench Pro v2 would be notable results for any model, but the relevant question for someone evaluating Flint is how it performs against other models that fit the same deployment envelope (a single B300 or H200), not against much larger hosted systems with far higher active-parameter counts.
Because exact comparison figures for competing local models weren’t detailed beyond the chart reference itself, the most defensible claim is this: Flint’s active parameter count (about 2.5B) is small relative to its total parameter count (39.8B), and its reported scores land in a range that would be considered strong for an open coding model at that active-parameter budget. Anyone benchmarking it against a specific competitor should run both under matched conditions, since the model card itself notes that “evaluation setups vary across sources.”
What do these scores mean for real-world coding tasks?
SWE-bench Verified and SWE-bench Pro v2 both test whether a model can resolve real GitHub issues by writing a patch that passes the project’s actual test suite, which makes them reasonable proxies for autonomous bug-fixing work. Terminal-Bench tests something more operational: can the model drive a terminal through multi-step tasks without going off the rails. A model scoring well on both suggests it can plausibly handle the two things Recursive says Flint is built for: investigating a codebase and fixing bugs or implementing features, and operating inside a terminal or tool-use loop without supervision at every step.
That said, benchmark scores are not a guarantee of performance on a specific codebase or task type. Recursive’s own limitations section is direct about this: the model can still produce incorrect or harmful code, faulty tool calls, or inappropriate actions, and it recommends testing generated code and reviewing consequential changes before applying them. Benchmarks tell you what a model can do under controlled, repeatable conditions; they don’t replace testing it on your own repository.
Frequently Asked Questions
What is Recursive Flint’s SWE-bench Verified score?
Flint scores 74.2% on SWE-bench Verified, as reported in its own model card evaluation section, using mini-swe-agent as the agent scaffold.
What is Recursive Flint’s Terminal-Bench score?
Flint scores 54.9% on Terminal-Bench 2.1, measured with the Terminus 2 harness. This score treats any solution reached via web retrieval as a failure, which is a stricter standard than some other reported Terminal-Bench scores use.
How many parameters does Flint have?
Flint has 39.8 billion total parameters in a mixture-of-experts architecture, but only about 2.5 billion are active per token (roughly 2.2 billion in the active body plus the output head), which is why it’s labeled “40B-A2.5B.”
What hardware does Flint need to run?
Recursive says one B300 or one H200 GPU per replica is sufficient to serve the BF16 checkpoint using its SGLang serving kit. The full checkpoint download is about 80 GB. Quantized versions for NVIDIA and Apple Silicon are distributed separately.
Are Flint’s benchmark scores comparable to other models’ reported numbers?
Only loosely. The model card notes that evaluation setups vary across sources, scores marked by Recursive were measured directly while others are developer-reported, and Flint applies a stricter rule on Terminal-Bench (penalizing web-retrieved answers) that other benchmarks may not apply.
