How to Run LFM2.5-24B-A2B Locally on Mac or GPU
Run LiquidAI's LFM2.5-24B-A2B locally via GGUF, MLX, or vLLM. Setup steps, hardware needs, and real throughput numbers from Liquid's own benchmarks.
What is LFM2.5-24B-A2B?
LFM2.5-24B-A2B is a mixture-of-experts (MoE) reasoning model from Liquid AI with 24 billion total parameters but only about 2.45 billion active per token. It’s the largest model in the LFM2.5 family, built specifically for agentic use: tool calling, multi-step reasoning, and running on local hardware rather than a cloud API. Because so few parameters activate per forward pass, it can run at usable speeds on a single GPU or even a Mac, something a dense 24B model couldn’t do.
TL;DR
- Sparse activation is the whole story: 24B total parameters but only 2.45B active per token means inference feels closer to a small dense model than a 24B one.
- Liquid ships every format you’d need: native weights for Transformers/vLLM/SGLang, an FP8 checkpoint for GPU serving, GGUF for llama.cpp, ONNX for cross-platform deployment, and an MLX 4-bit build for Apple Silicon.
- Apple Silicon numbers are public: Liquid reports 184 tokens/sec on an Apple M5 Max, which is fast enough for interactive agent workloads on a laptop.
- Datacenter GPUs scale way up: on a single NVIDIA B300, Liquid reports up to 21,000 output tokens/sec, aimed at high-throughput serving rather than single-user chat.
- It always reasons first: the chat template inserts a
<think>tag automatically, so every response includes a reasoning trace before the final answer, unlike the earlier non-reasoning LFM2-24B-A2B. - It’s a tool-use specialist, not a knowledge base: Liquid explicitly recommends pairing it with retrieval or web search rather than relying on it for raw facts.
- Context goes up to 131,072 tokens, quadrupled from the prior model’s 32K, with a doubled 128K vocabulary for better multilingual and non-Latin tokenization.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
Why does the “A2B” part matter for local deployment?
The “A2B” in the name stands for “2B active.” LFM2.5-24B-A2B uses 64 experts per MoE block with top-4 routing, so each token only passes through a small slice of the full network. That’s the mechanism behind Liquid’s claim of running up to 1.7x faster than comparable MoE models on-device. For anyone trying to run large language models locally, this is the practical difference between “needs a server rack” and “runs on a laptop.” You still need enough memory to hold all 24B parameters (or a quantized version of them), but the compute cost per generated token tracks closer to a 2B-3B model.
The architecture itself carries over from LFM2-24B-A2B: 40 layers made up of 30 gated short-convolution blocks and 10 grouped-query attention (GQA) blocks. What changed for the 2.5 generation is context length (32K to 131K) and vocabulary size (65K to 128K tokens), plus a training run that nearly doubled in scale to around 33 trillion pre-training tokens.
Which quantized formats are available, and which should you pick?
Liquid publishes five deployment variants on Hugging Face:
- LFM2.5-24B-A2B: the original bfloat16 checkpoint. Best for fine-tuning or serving with Transformers, vLLM, or SGLang.
- LFM2.5-24B-A2B-FP8: block-wise FP8 weights with dynamic FP8 activations. Built for high-throughput GPU serving with lower memory footprint than full precision.
- LFM2.5-24B-A2B-GGUF: quantized for llama.cpp and compatible runtimes (including LM Studio). This is the format most people running the model on a personal machine without a beefy GPU will want, since llama.cpp handles CPU offloading well.
- LFM2.5-24B-A2B-ONNX: for cross-platform deployment across cloud, edge, and mobile environments via ONNX Runtime.
- LFM2.5-24B-A2B-MLX-4bit: a 4-bit quantization built for Apple’s MLX framework, specifically tuned for Apple Silicon.
A DSpark draft model for speculative decoding with SGLang is listed as “coming soon,” which would speed up generation further on supported GPU setups.
Picking a format comes down to hardware: GGUF for CPU-heavy or mixed CPU/GPU Mac and PC setups, MLX for Apple Silicon users who want the most optimized path, FP8 for single-GPU servers that want throughput, and the native checkpoint for anyone fine-tuning or running on vLLM/SGLang in production.
How fast does it actually run on a Mac?
Liquid’s own benchmark reports 184 tokens per second on an Apple M5 Max using the model. That number comes from Liquid’s published figures for the MLX build, which is the path Apple Silicon users should take if they want the fastest local inference, since MLX is built to use the unified memory and matrix units on M-series chips more efficiently than generic CPU inference. For a 24B-parameter model, that throughput is well into comfortable range for interactive agent use and chat, not just batch jobs.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
On the GPU side, Liquid reports up to 21,000 output tokens per second on a single NVIDIA B300, a figure aimed at high-throughput server deployments rather than individual chat sessions. That gap between a laptop number (184 tok/s) and a datacenter GPU number (21K tok/s) illustrates the range this model is built to cover, from on-device agents to production API serving, using the same weights.
How do you set it up with llama.cpp or MLX?
For llama.cpp: download the LFM2.5-24B-A2B-GGUF checkpoint, pick a quantization level that fits your RAM, and run it through a standard llama.cpp build or a front end like LM Studio, which Liquid explicitly lists as a supported runtime. llama.cpp handles CPU offloading, so you can split layers between GPU VRAM and system RAM if you don’t have enough VRAM to hold the whole model.
For MLX on Apple Silicon: grab the LFM2.5-24B-A2B-MLX-4bit build and run it with the MLX framework, which is optimized specifically for M-series chips. This is the path that produced Liquid’s 184 tok/s figure on an M5 Max.
For GPU serving at scale, vLLM and SGLang both support the model directly from the native or FP8 checkpoints, and Liquid provides Colab notebooks for Transformers, vLLM, and llama.cpp inference to get started without local setup first.
Whichever backend you use, once it’s running an OpenAI-compatible server (llama.cpp and vLLM both support this), you can point agent harnesses like Hermes, OpenClaw, or Pi at it by setting the base URL to your local endpoint, as documented in Liquid’s model card with ready-made config snippets for each.
Is it worth running locally instead of calling a cloud API?
If your use case is an agent that needs to call tools, operate offline, or avoid sending data to a third party, this model is purpose-built for that. Liquid trained it directly inside popular agent harnesses (OpenClaw, Hermes Agent, Pi, OpenCode) during reinforcement learning, so it already understands their tool formats and system prompt conventions out of the box. On agentic and tool-use benchmarks, Liquid reports it scoring 48.09 on LiveClaw versus 38.19 for Qwen3.6-35B-A3B, a model with roughly 50% more total and active parameters.
Where it’s weaker is pure knowledge recall. Liquid’s own documentation says it isn’t recommended for knowledge-heavy tasks on its own, and suggests pairing it with retrieval or web search tools instead, which plays to its tool-calling strength rather than asking it to be an encyclopedia.
For anyone prioritizing privacy, offline operation, or low per-token cost over raw benchmark-topping knowledge, running it locally through GGUF or MLX is a reasonable tradeoff. For teams that need maximum throughput and don’t mind cloud GPU costs, the FP8 checkpoint on vLLM or SGLang is the better fit.
Frequently Asked Questions
What hardware do I need to run LFM2.5-24B-A2B locally?
Liquid demonstrates it running on an Apple M5 Max laptop (184 tok/s via MLX) and on a single NVIDIA B300 GPU (up to 21,000 tok/s via vLLM/SGLang with FP8). Between those extremes, quantized GGUF builds let it run on more modest CPU/GPU combinations, though exact VRAM or RAM requirements for each quantization level aren’t specified in Liquid’s model card.
Does LFM2.5-24B-A2B always show its reasoning?
Yes. Unlike the earlier LFM2-24B-A2B, this is a pure reasoning model. Its chat template automatically inserts a <think> tag before the assistant’s answer, so it always reasons before responding.
Remy is new. The platform isn't.
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
What’s the difference between the GGUF and MLX versions?
GGUF is built for llama.cpp and compatible tools like LM Studio, optimized for CPU inference and cross-platform use. The MLX 4-bit build is specifically optimized for Apple’s MLX framework and Apple Silicon hardware, and is the version behind Liquid’s published Mac performance numbers.
Can I fine-tune LFM2.5-24B-A2B?
Yes. Liquid provides Colab notebooks for continued pre-training and supervised fine-tuning (with LoRA) using both Unsloth and Hugging Face’s TRL library, covering text completion, translation, and general SFT workflows.
How does it compare to similarly sized models like Qwen3.6-35B-A3B?
On Liquid’s reported LiveClaw agentic benchmark, LFM2.5-24B-A2B scores 48.09 versus 38.19 for Qwen3.6-35B-A3B, despite Qwen’s roughly 50% larger total and active parameter counts. Liquid also claims it runs up to 1.7x faster than comparable MoE models on-device.