LFM2.5-24B-A2B Benchmarks: How It Beats Bigger MoE Models
LFM2.5-24B-A2B edges out larger MoE rivals on agentic and tool-use benchmarks while using far fewer active parameters per token.
What is LFM2.5-24B-A2B and why does it matter?
LFM2.5-24B-A2B is a 24-billion-parameter mixture-of-experts (MoE) model from Liquid AI that activates only about 2.45 billion parameters per token. It’s the largest model in the LFM2.5 family, a lineup that now ranges from a 230-million-parameter model up to this one. The pitch is simple: get agent and tool-use performance close to (or better than) models with 50% more active and total parameters, while running fast enough to live on a laptop or a single GPU. It matters because most “agentic” benchmarks today are won by brute-force scaling. LFM2.5-24B-A2B is a data point for the opposite approach: better training and routing instead of more active compute per token.
TL;DR
- LFM2.5-24B-A2B has 24B total parameters but only 2.45B active per token, using 64 experts per MoE block with top-4 routing across 40 layers.
- On LiveClaw, a benchmark for agentic and tool-use tasks, it scores 48.09 versus 38.19 for Qwen3.6-35B-A3B, a model with roughly 50% more total and active parameters.
- The model is a pure reasoning model: it always generates a
<think>block before answering, unlike its predecessor LFM2-24B-A2B. - Context length jumped from 32K to 131,072 tokens, and the vocabulary doubled from 65K to 128K tokens to improve non-Latin language tokenization.
- Inference is tuned for on-device speed: 184 tokens/sec on an Apple M5 Max, and up to 21,000 output tokens/sec on a single Nvidia B300 GPU, claimed to be up to 1.7x faster than comparable MoEs.
- Post-training leaned heavily on agentic reinforcement learning inside real agent harnesses (OpenClaw, Hermes Agent, Pi, OpenCode), so the model learns actual tool-call formats and system prompt conventions, not just synthetic tool-use data.
- Liquid AI explicitly recommends pairing it with retrieval or web search rather than relying on it for raw knowledge recall, positioning it as a tool-use specialist first.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
How does LFM2.5-24B-A2B compare to Qwen3.6-35B-A3B?
The headline comparison in Liquid AI’s release is against Qwen3.6-35B-A3B, a MoE model with both more total parameters (35B vs 24B) and more active parameters per token. On LiveClaw, a benchmark built around agentic and tool-calling behavior, LFM2.5-24B-A2B scores 48.09 against Qwen’s 38.19, a meaningful gap in favor of the smaller-active model. Liquid frames this as “best-in-class agent” performance relative to a model with roughly 50% more total and active parameters.
This is a narrower, task-specific claim rather than a general “beats Qwen at everything” statement. The comparison is scoped to agentic and tool-use tasks, which is where Liquid has clearly concentrated its post-training effort. The model card is explicit that LFM2.5-24B-A2B is not recommended as a standalone knowledge engine, so broader reasoning or world-knowledge benchmarks aren’t the basis for this comparison.
What is AutomationBench and why does it matter for agent models?
AutomationBench and LiveClaw both sit in a newer category of benchmarks built to test whether a model can actually operate tools and complete multi-step tasks, not just answer questions. Instead of scoring a model on multiple-choice knowledge or static reasoning puzzles, these benchmarks evaluate whether a model can parse a tool schema, make a correctly formatted function call, interpret the result, and produce a coherent final answer, sometimes across multiple turns.
For builders, this distinction matters more than traditional benchmarks like MMLU. A model can score well on general knowledge tests and still fail badly at agent tasks because it hallucinates tool arguments, loses track of state across turns, or doesn’t follow a harness’s expected calling convention. LFM2.5-24B-A2B was specifically post-trained inside real agent harnesses, including OpenClaw, Hermes Agent, Pi, and OpenCode, which is likely why it performs disproportionately well on these task types relative to its active parameter count.
How does the architecture achieve this efficiency?
The model keeps the same core architecture as LFM2-24B-A2B: 40 layers split between 30 double-gated short convolution blocks and 10 grouped-query attention (GQA) blocks, with MoE blocks containing 64 experts and top-4 routing. Two things changed for the 2.5 version: context length grew from 32K to 131,072 tokens, and the tokenizer vocabulary doubled from 65,000 to 128,000 tokens through what Liquid calls an “in-place tokenizer expansion,” aimed at improving tokenization efficiency for non-Latin scripts across the model’s 16 supported languages.
The training budget also scaled up substantially, from roughly 17 trillion to about 33 trillion pre-training tokens, with a combined training budget of 36 trillion tokens once mid-training and long-context phases are included. The mid-training phase used 2 trillion tokens at 32K context weighted toward reasoning, math, code, instruction following, and multilingual data, followed by 300 billion tokens of long-context training at the full 131K window. Tool calls and multi-turn agent trajectories were present in both phases, not bolted on afterward.
Post-training combined supervised fine-tuning, teacher specialization, two stages of multi-domain on-policy distillation (one general, one agentic), and agentic reinforcement learning run inside the agent harnesses mentioned above. That last step is the most distinctive part: rather than training tool-use behavior purely from static examples, the model practiced inside actual agent loops and learned harness-specific system prompts and interaction patterns directly.
Is LFM2.5-24B-A2B a reasoning model?
Yes, and this is a change from its predecessor. LFM2.5-24B-A2B always generates a <think> block before producing a final answer. The chat template inserts the <think> tag automatically at the start of the assistant’s turn, so every response includes a visible reasoning trace by default. Liquid describes this as “reasoning at MoE speed”: because only 2.45B parameters are active per token, the cost of generating those extra reasoning tokens stays low compared to a dense model or a MoE with a larger active parameter count.
This design choice ties back to the agentic framing. Multi-step tool use benefits from a model that reasons about which tool to call and why before committing to a function call, rather than jumping straight to an action. The tradeoff is that every response, even simple ones, carries the overhead of a reasoning block, which is why raw throughput (tokens per second) becomes an important practical metric alongside benchmark scores.
What are the practical deployment numbers?
Liquid AI reports 184 tokens/sec on an Apple M5 Max and up to 21,000 output tokens/sec on a single Nvidia B300 GPU, with a claim of up to 1.7x faster inference than comparable MoE models on-device. The model ships in several formats to match these use cases: the native checkpoint for Transformers, vLLM, and SGLang; an FP8 checkpoint for high-throughput GPU serving with lower memory use; a GGUF build for llama.cpp and CPU inference; an ONNX build for cross-platform and mobile deployment; and an MLX 4-bit build specifically for Apple Silicon. A DSpark draft model for speculative decoding with SGLang was listed as “coming soon” at release.
Tool calling uses a Pythonic format by default (a Python-style list between special tool-call tokens), though the model can be instructed via system prompt to output JSON tool calls instead. Liquid provides direct configuration snippets for connecting the model to Hermes, OpenClaw, and Pi agent harnesses over an OpenAI-compatible local API, which lowers the integration barrier for anyone already using those tools.
Is LFM2.5-24B-A2B worth using over a larger model?
It depends on the task. For agentic workflows, tool calling, and on-device deployment where latency and memory matter, the benchmark numbers suggest LFM2.5-24B-A2B is a strong pick relative to its resource footprint, especially against larger-active-parameter MoE models like Qwen3.6-35B-A3B on agent-specific benchmarks. For knowledge-heavy tasks that depend on the model’s own stored information rather than tool use, Liquid’s own guidance is to pair it with retrieval or web search rather than rely on it standalone. That’s a reasonable constraint given its relatively compact active parameter count, and it reflects a model designed around a specific job (reasoning plus tool execution) rather than as a general-purpose knowledge base.
Frequently Asked Questions
What does “A2B” mean in LFM2.5-24B-A2B?
It refers to the active parameter count: the model has 24 billion total parameters but only about 2 billion (2.45B specifically) are active per token, thanks to its mixture-of-experts routing with 64 experts and top-4 selection.
One coffee. One working app.
You bring the idea. Remy manages the project.
How does LFM2.5-24B-A2B compare to Gemma models?
The provided benchmark data centers on the Qwen3.6-35B-A3B comparison via LiveClaw; no specific Gemma benchmark figures are included in Liquid’s release materials, so direct numeric comparisons to Gemma can’t be confirmed from available sources.
Can LFM2.5-24B-A2B run on a laptop?
Yes. Liquid provides an MLX 4-bit build for Apple Silicon and reports 184 tokens/sec on an Apple M5 Max, along with GGUF support for llama.cpp and CPU inference, making local deployment feasible on consumer hardware.
Does LFM2.5-24B-A2B always show its reasoning?
Yes. Unlike the earlier LFM2-24B-A2B, this model always inserts a <think> block before its final answer, since the chat template automatically starts assistant turns with that tag.
What’s the context window for LFM2.5-24B-A2B?
131,072 tokens, up from 32,000 in the earlier LFM2-24B-A2B, achieved through additional long-context training on 300 billion tokens.
