How to Run LFM2.5-24B-A2B Locally for On-Device Agents
A practical guide to running Liquid AI's LFM2.5-24B-A2B mixture-of-experts model locally via Transformers, llama.cpp, MLX, and vLLM.
What is LFM2.5-24B-A2B and why does it matter for local AI agents?
LFM2.5-24B-A2B is a 24-billion-parameter mixture-of-experts (MoE) model from Liquid AI that only activates about 2.45 billion parameters per token. That gap between total size and active compute is the whole point: you get the knowledge capacity of a much bigger model while paying the inference cost of a small one. It’s built specifically for agentic use, meaning tool calling, reasoning, and running inside agent harnesses on a laptop, workstation, or single GPU rather than a data center cluster.
It’s the largest model in Liquid’s LFM2.5 family, which ranges from a 230-million-parameter model up to this 24B-A2B flagship. Unlike its predecessor LFM2-24B-A2B, this version is a “pure reasoning model”: it always generates a <think> block before answering, which is baked directly into its chat template.
TL;DR
- LFM2.5-24B-A2B uses a mixture-of-experts design with 64 experts per MoE block and top-4 routing, giving it 24B total parameters but only 2.45B active per token.
- The model reports 48.09 on LiveClaw versus 38.19 for Qwen3.6-35B-A3B, a model with roughly 50% more total and active parameters, on agentic and tool-use benchmarks.
- On-device throughput is a headline feature: Liquid reports 184 tokens per second on an Apple M5 Max and up to 21,000 output tokens per second on a single NVIDIA B300 GPU.
- The architecture combines 30 double-gated short convolution blocks with 10 grouped-query attention (GQA) layers across 40 total layers, trained on 36 trillion tokens with a 131,072-token context window.
- Liquid ships the model in five formats, native safetensors, FP8, GGUF, ONNX, and MLX 4-bit, covering Transformers, vLLM, SGLang, llama.cpp, and Apple Silicon out of the box.
- Tool calling is native to the model, using Pythonic function-call syntax by default, with direct integration recipes for agent harnesses like OpenClaw, Hermes, and Pi.
- Liquid explicitly recommends pairing it with retrieval or web search rather than relying on it for knowledge-heavy tasks, since its strength is tool use and reasoning, not raw memorized facts.
How does the mixture-of-experts architecture keep it fast?
The model has 40 layers total: 30 are “double-gated short convolution blocks” (Liquid’s LFM2 architecture signature) and 10 use grouped-query attention. Each MoE block routes tokens to 4 of 64 available experts, so despite the 24B parameter count sitting on disk and in memory, each forward pass only touches about 2.45B parameters worth of compute.
This is why the model can be competitive with Qwen3.6-35B-A3B, a model with roughly 50% more total and active parameters, while running noticeably faster. Liquid claims up to 1.7x faster inference than comparable MoE models on-device. The context window is also large for a model this size: 131,072 tokens, up from 32K in the earlier LFM2-24B-A2B, thanks to a dedicated long-context training phase.
The vocabulary was expanded too, doubling from 65K to 128K tokens through what Liquid calls “in-place tokenizer expansion,” aimed at improving tokenization efficiency for non-Latin scripts. The model supports 16 languages including Arabic, Chinese, Japanese, Korean, Hindi, and Russian.
What hardware do you need to run it locally?
Because only 2.45B parameters are active per token, the compute demand per step is light, but you still need enough memory to hold all 24B parameters (or a quantized version of them) resident. That makes format choice the main lever for fitting the model onto your hardware:
- GGUF (quantized): Built for llama.cpp and CPU-friendly deployment. This is the easiest path for running on a laptop or desktop without a high-end GPU, with reduced memory footprint depending on quantization level.
- MLX 4-bit: Purpose-built for Apple Silicon. Liquid reports 184 tokens per second on an Apple M5 Max, which puts real-time agentic use within reach on a MacBook Pro-class machine.
- FP8: Block-wise FP8 weights with dynamic FP8 activations, aimed at GPU serving through vLLM or SGLang with a smaller memory footprint than full precision.
- Native safetensors (bfloat16): The full-precision checkpoint, best for fine-tuning or when running with Transformers, vLLM, and SGLang on a capable GPU.
- ONNX: Cross-platform format for hardware-accelerated inference across cloud, edge, and mobile targets.
On the high end, Liquid reports up to 21,000 output tokens per second on a single NVIDIA B300 GPU, which is squarely a data-center figure, not something to expect from local hardware. The practical takeaway for anyone building on a single machine: GGUF on CPU or MLX on a Mac are the realistic entry points, while FP8 and native weights target GPU servers.
How do you set it up with Transformers, llama.cpp, MLX, and vLLM?
Liquid publishes direct recipes for each backend. The simplest is Transformers (requiring transformers>=5.0.0):
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
model_id = "LiquidAI/LFM2.5-24B-A2B"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
dtype="bfloat16",
)
tokenizer = AutoTokenizer.from_pretrained(model_id)Generation uses specific recommended settings: temperature 0.1, top_k 50, repetition penalty 1.05. These aren’t arbitrary. Liquid bakes them into the generation config, and since the model always reasons before answering, a low temperature keeps the <think> phase from wandering before the final answer.
For CPU-first or constrained hardware, llama.cpp with the GGUF checkpoint is the path, running cross-platform with CPU offloading. For Apple Silicon, the MLX 4-bit build is optimized specifically for Mac hardware. For production-style GPU serving with high throughput, vLLM or SGLang with the FP8 checkpoint is the recommended route.
Whichever backend you pick, the model exposes an OpenAI-compatible API once served, which is what makes the agent harness integrations possible.
Is it worth running locally for agent workloads specifically?
The model’s post-training was explicitly built around agentic use, not general chat. Liquid’s training recipe includes two on-policy distillation stages (one non-agentic, one agentic) plus agentic reinforcement learning run inside real agent harnesses: OpenClaw, Hermes Agent, Pi, and OpenCode. That means the model learned each harness’s actual tool definitions, system prompts, and interaction patterns, rather than being trained on synthetic tool-call examples and hoping it generalizes.
Tool calling itself defaults to Pythonic syntax (a Python-style list wrapped in <|tool_call_start|> and <|tool_call_end|> tokens), though you can instruct it to emit JSON instead. The four-step flow, function definition, function call, function execution, final answer, is handled through the standard chat template and tools= argument in apply_chat_template().
Liquid is also direct about where this model isn’t a good fit: it’s “not recommended for knowledge-heavy tasks on its own.” The architecture trades broad memorized knowledge for routing efficiency and tool-use skill, so the intended pattern is pairing it with retrieval or web search for facts, while leaning on the model for reasoning and orchestration. For a local agent that needs to call APIs, browse, or operate a coding harness, that tradeoff lines up well. For a model you’d ask trivia questions, less so.
Frequently Asked Questions
How much VRAM or memory does LFM2.5-24B-A2B need?
Liquid doesn’t publish a single required-memory figure, but the model has 24B total parameters, so memory needs scale with which checkpoint you use. The GGUF quantized builds and MLX 4-bit build are designed to shrink the footprint substantially for CPU or Apple Silicon use, while the native bfloat16 and FP8 checkpoints target GPUs with more headroom.
Can LFM2.5-24B-A2B run on a MacBook?
Yes. Liquid provides a dedicated MLX 4-bit build for Apple Silicon and reports 184 tokens per second on an Apple M5 Max, which supports responsive local agent use on recent Mac hardware.
What’s the difference between LFM2.5-24B-A2B and LFM2-24B-A2B?
The 2.5 version extends context length from 32K to 131K tokens, doubles the vocabulary from 65K to 128K tokens for better multilingual tokenization, and switches to a pure reasoning model that always generates a <think> block before its final answer.
Does it support function calling out of the box?
Yes. The model was trained with tool use in both its mid-training and post-training phases, including reinforcement learning inside agent harnesses like OpenClaw and Hermes Agent. It outputs Pythonic function calls by default, with an option to switch to JSON via the system prompt.
Which deployment format should I choose?
Use GGUF with llama.cpp for CPU or general local deployment, MLX 4-bit if you’re on a Mac, FP8 for high-throughput GPU serving with vLLM or SGLang, and the native safetensors checkpoint if you plan to fine-tune the model.
