How to Run Flint Locally: Ollama, llama.cpp, and Rec Setup Guide

What is Flint and why would you run it locally?

Flint is an open-weight coding model from Recursive, built specifically to drive agentic coding tasks like fixing bugs, writing tests, and implementing features. It’s a mixture-of-experts model with 39.8 billion total parameters, but only about 2.5 billion are active per token, which is why it runs at usable speeds on consumer hardware instead of requiring a data center GPU. The weights are released under Apache 2.0, so there’s no API cost and no usage restriction. You can run it on llama.cpp, Ollama, LM Studio, vLLM, SGLang, or Recursive’s own inference engine, Rec.

TL;DR

Flint is a mixture-of-experts model (39.8B total, ~2.5B active parameters per token) tuned specifically for coding agents, not general chat or creative writing.
The official BF16 checkpoint needs serious data-center hardware (one B300 or H200 GPU, roughly 80GB download), but quantized GGUF builds make it realistic to run on a single high-end laptop or desktop GPU.
Recursive’s own inference engine, Rec, is the fastest option but currently only supports specific Apple Silicon chips (M4 Pro, M5 Pro, M5 Max), notably excluding the M4 Max.
On llama.cpp, real-world speed drops sharply as context grows: roughly 110 tokens/sec on short prompts, down to around 9 tokens/sec past 100,000 tokens of context.
Running Flint through Ollama on a desktop GPU setup consumed just over 32GB of VRAM, with headroom to shrink that further by reducing KV cache size.
Flint pairs with the Pi and OpenCode coding harnesses, both of which Recursive explicitly recommends, and both need a Hugging Face config file dropped into place to register the model.
Benchmark claims put Flint ahead of similarly sized mixture-of-experts models on SWE-bench Verified, SWE-bench Pro, and Terminal-Bench, while trailing larger dense models like Qwen3 on the hardest tasks.

REMY IS NOT

✕a coding agent
✕no-code
✕vibe coding
✕a faster Cursor

IT IS

✓a general contractor for software

The one that tells the coding agents what to build.

What hardware do you need to run Flint?

The requirements depend heavily on which checkpoint you use.

The full BF16 checkpoint is meant for server-grade GPUs. Recursive’s documentation specifies one B300 or H200 GPU per replica, served through an SGLang-based Docker setup, with an initial download of around 80GB.

For local use, the practical path is a quantized GGUF. The int4 GGUF release (hosted at RecursiveSI/flint-int4-gguf on Hugging Face) is about 26GB. Recursive’s own guidance for running Flint through its Rec engine specifies a minimum of 48GB of unified memory on Apple Silicon. Testing on an M4 Max with 48GB landed right at that ceiling. On the GPU side, running the quantized model through Ollama consumed just over 32GB of VRAM, a figure that can be reduced further by shrinking the KV cache. If you want to run the unquantized model on a single GPU rather than a multi-GPU server setup, plan on needing more than 80GB of VRAM, something out of reach for most individual setups.

Context length is another factor that affects memory and speed together. Flint’s documented context window is 262,144 tokens, though Recursive recommends staying under 256,000 in practice. Coding agents tend to front-load large prompts. Initial prompts from Pi and OpenCode have been observed in the 17,000 to 26,000 token range before any actual work begins, meaning you’re often already deep into the “long context” performance zone from the first turn.

How do you set up Flint with Ollama?

Ollama is the simplest entry point if you’re not on Apple Silicon or don’t want to compile anything.

Install the latest version of Ollama.
Pull the model with ollama pull, specifying the Flint model tag.
Confirm it’s loaded with ollama list, and check VRAM usage with your GPU’s monitoring tool (expect roughly 32GB for the quantized build).
If VRAM is tight, reduce the KV cache size in Ollama’s runtime settings to bring memory use down further.

This is the path of least resistance, but it won’t match the speed of a dedicated inference runtime tuned for the model’s architecture.

How do you set up Flint with llama.cpp?

llama.cpp support for Flint required a custom build in the early release window, since the architecture support landed first as an open pull request rather than in a stable release. If you’re setting this up after that PR has merged, a standard update should work; otherwise you’ll need to build from the relevant branch yourself.

Steps:

Build llama.cpp from the branch with Flint support (or update normally if it’s since been merged).
Download the int4 GGUF from the RecursiveSI/flint-int4-gguf Hugging Face repo (about 26GB). The repo also includes two small config files for the Pi and OpenCode harnesses.
Start the server with llama-server, pointing to the model file, with the --jinja flag enabled for chat templating.
Apply Recursive’s recommended sampling settings: temperature 1.0, top-p 0.95, top-k 20. The full command is documented on the model card.

Expect speed to vary heavily with context length. In testing, llama.cpp output speed on short prompts ran around 110 tokens per second, with prompt processing around 300 tokens per second. At 8,000 tokens of context that output speed fell to roughly 52 tokens per second, and at 32,000 tokens it dropped to around 32 tokens per second. In longer agent sessions where context grew past 100,000 tokens, speed fell as low as 9 tokens per second. Note that the llama.cpp PR implementation was reported by its author to carry a 10 to 30% slowdown versus the eventual merged version, so these numbers likely improve over time.

How does Rec, Recursive’s own engine, compare?

Rec is Recursive’s purpose-built inference engine for Apple Silicon, and it’s where most of the speed advantage shows up. Recursive claims it runs 1.4 to 2.6 times faster than llama.cpp, achieved by fusing the decode pass into custom metal kernels tuned per chip. Recursive’s published numbers cite more than 125 tokens per second on an M4 Pro MacBook Pro and more than 150 tokens per second on an RTX Spark system.

Installation is a single curl command, followed by rec launch pi, which downloads the model, starts the server, and launches the Pi coding harness automatically.

The catch: Rec’s custom kernels are chip-specific, and as of this release they only support the M4 Pro, M5 Pro, and M5 Max. Notably, the M4 Max, despite being a faster chip overall, isn’t supported, because Rec’s kernels need individual tuning per chip rather than working generically across Apple Silicon. If your Mac isn’t on that supported list, llama.cpp is the fallback, with the speed tradeoffs described above.

How do you connect Flint to a coding agent?

Flint is designed to be the “brain” behind an agent harness rather than used through raw chat. Recursive specifically recommends two harnesses: Pi and OpenCode.

For Pi, copy the Pi config file from the Hugging Face repo into your Pi agent folder (typically ~/.pi/agent/) as models.json.

For OpenCode, add the Flint provider block from the published OpenCode config into your own OpenCode configuration file.

One configuration detail matters here: OpenCode’s default behavior is to stop waiting for a model response after 5 minutes of silence. Because local inference sends no output until it finishes processing the entire prompt, a 26,000-token prompt can take long enough to process that OpenCode times out and retries, effectively looping without producing anything. The fix is to set both headerTimeout and timeout to false in the Flint provider options inside your OpenCode config.

Once configured, Flint should appear automatically in the model picker of any interface that reads from Pi or OpenCode’s configuration, without needing separate setup inside that interface.

Is Flint worth running locally?

For focused, scoped coding tasks, yes. In testing, Flint performed well on tasks like fixing bugs in an existing codebase against a test suite, and building small backend APIs with validation, rate limiting, and safe file writes. It also handled front-end generation reasonably, though with occasional small bugs (like unsized icon elements breaking a layout) that needed a manual fix.

Where it struggles is scope creep and long context. On tasks where the agent searched broadly through unrelated files or let conversation context balloon past 100,000 tokens, both quality of experience and raw speed suffered. Recursive’s own benchmark claims position Flint as the strongest mixture-of-experts model under 50 billion parameters on SWE-bench Verified, SWE-bench Pro, and Terminal-Bench, while acknowledging that larger dense models still win on the hardest software engineering and terminal tasks.

It’s also explicitly not a general-purpose or creative model. Attempts to use it for translation or open-ended creative writing produced mediocre results, which lines up with Recursive’s own framing of Flint as a coding-only tool.

Frequently Asked Questions

How much VRAM does Flint need to run locally?

The quantized int4 build used roughly 32GB of VRAM through Ollama, reducible by lowering the KV cache. Running through Rec on Apple Silicon requires at least 48GB of unified memory. The full BF16 checkpoint needs a single B300 or H200 GPU, or upward of 80GB of VRAM if squeezed onto one card.

Does Flint work with Ollama and LM Studio?

Yes. Flint supports Ollama, LM Studio, llama.cpp, vLLM, and SGLang, in addition to Recursive’s own Rec engine.

Why doesn’t Rec support the M4 Max?

Rec uses custom kernels tuned individually per Apple Silicon chip. At release, only the M4 Pro, M5 Pro, and M5 Max were supported, meaning even faster chips like the M4 Max fall back to llama.cpp until Recursive adds support.

What context length does Flint support?

The documented maximum is 262,144 tokens, but Recursive recommends staying under 256,000 for reliable performance, since speed and quality both degrade as context grows.

Can Flint be used for things other than coding?

It’s built and tuned specifically for agentic coding tasks like debugging, feature implementation, and test writing. It performs poorly on translation and produces only average results on open-ended creative writing tasks.