LFM3-1.7B-Base: LiquidAI's Tiny Model for Phones and Laptops
LFM3-1.7B-Base runs in under 1GB at 4-bit, handles 256K context, and uses a tri-hybrid architecture built for local, offline AI.
What is LFM3-1.7B-Base?
LFM3-1.7B-Base is a 1.73-billion-parameter language model from Liquid AI, released as the pre-trained checkpoint behind the company’s LFM3-1.7B chat model. It was trained on 22 trillion tokens, supports a 256K-token context window, and runs in roughly 1GB of memory at 4-bit quantization, small enough to fit on a phone or laptop. It’s built on a “tri-hybrid” architecture mixing convolution and attention layers, and it’s meant as a starting point for fine-tuning rather than a drop-in chat assistant.
TL;DR
- LFM3-1.7B-Base is the raw pre-trained checkpoint underlying LiquidAI’s LFM3-1.7B chat model, intended mainly for continued pre-training and custom fine-tuning rather than direct chat use.
- The model holds 1.73 billion parameters and was trained on 22 trillion tokens across a two-stage pipeline: broad pre-training followed by curated mid-training on agentic data.
- It supports a 256K-token context window (input plus output), reached through staged extensions late in training rather than from the start.
- At 4-bit quantization it needs about 1GB of memory, which puts it within reach of phones, laptops, and other memory-constrained edge devices.
- The architecture is a 24-layer tri-hybrid design: 13 gated short convolution layers, 5 sliding-window attention layers, and 6 global attention layers, mixing cheap local processing with full-context attention.
- It handles 16 languages including English, Arabic, Chinese, French, German, and Hindi, with a 128,000-token vocabulary.
- The model ships under Liquid AI’s own LFM Open License v1.0, and its sibling, LFM3-1.7B, adds chat formatting, tool calling, and agentic post-training on top of this same base.
What makes LFM3-1.7B-Base different from a typical small LLM?
Most small open models in this size class are either general chat models shrunk down or base models meant purely for research. LFM3-1.7B-Base sits in between. It’s explicitly positioned as unfinished: Liquid AI recommends it only for use cases that require heavy fine-tuning, like building a language-specific assistant, training on proprietary internal data, or experimenting with new post-training methods. If you just want a chatbot or tool-calling agent, the companion release, LFM3-1.7B, is the one to use, since it already went through supervised fine-tuning, mixed reinforcement learning, multi-domain on-policy distillation, and agentic RL on top of this base.
What separates LFM3-1.7B-Base from a typical transformer base model is the architecture underneath it, which trades some of the standard full-attention design for something built around efficiency on constrained hardware.
How does the tri-hybrid architecture work?
The model has 24 layers split three ways:
- 13 gated short convolution layers, which process local patterns in the sequence cheaply, without the quadratic cost of attention.
- 5 sliding-window attention layers, which attend to a limited local window around each token.
- 6 global attention layers, which handle long-range dependencies across the full context.
This mix is why Liquid AI calls it “tri-hybrid” rather than a pure transformer. The convolution layers do the bulk of the sequence processing cheaply, the sliding-window layers add mid-range context awareness, and the global attention layers are reserved for the places where full-sequence reasoning actually matters. The result is a model that can scale to very long contexts (256K tokens) without the memory and compute costs scaling the way they would in an all-attention design. This approach follows a pattern familiar from other efficient-architecture research: use attention sparingly, lean on convolution where possible.
How does it reach a 256K-token context window?
Context length wasn’t baked in from the start. The published training recipe shows a staged buildup:
- Main pre-training, around 15 trillion tokens of web and synthetic data at shorter context.
- Code, math, and science training, roughly 3 trillion tokens at 4K context.
- Mid-training, about 3 trillion tokens of curated agentic data (traces and tool-call examples) at 32K context.
- 128K extension, roughly 500 billion tokens.
- 256K extension, roughly 50 billion tokens.
That adds up to the full 22 trillion training tokens. The practical implication is that long-context ability was deliberately extended late in training through dedicated stages, rather than emerging from training on long documents throughout. The last 3.5 trillion tokens overall are described as higher-quality, curated data with a heavier mix of agentic traces and tool-calling examples, which is also what sets up the base checkpoint for agentic fine-tuning later.
Why does under-1GB memory matter for edge deployment?
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
Running a capable language model locally, on a phone, laptop, or embedded device, without a network connection or a GPU server, has been a persistent bottleneck for on-device AI. Model size at a given precision directly determines whether that’s feasible. At 4-bit quantization, LFM3-1.7B-Base needs about 1GB of memory, which fits comfortably within the RAM budgets of modern phones and ordinary laptops. That opens up use cases where sending data to a cloud API isn’t acceptable or practical: offline assistants, privacy-sensitive applications, embedded devices, or scenarios with unreliable connectivity.
The tradeoff is capability. A 1.7B-parameter model, however efficiently trained, will not match a large frontier model on broad reasoning or knowledge-heavy tasks. The value proposition here is narrower: a small footprint model meant to be specialized for a specific domain, language, or task, and run entirely on local hardware, rather than a general-purpose assistant competing with much larger systems.
Is LFM3-1.7B-Base ready to use out of the box?
Not for most people. It’s a base checkpoint, trained for next-token prediction with some chat-formatted mid-training data mixed in, but it hasn’t been through the full post-training process that makes a model reliably follow instructions or hold a conversation. The model card is explicit that it’s “only recommended for tasks that require heavy fine-tuning.” For anything resembling chat, tool use, or agent workflows, Liquid AI points users to LFM3-1.7B instead, which applies supervised fine-tuning and reinforcement learning on top of this same base.
That said, the base checkpoint does support inference directly through Hugging Face’s Transformers library with BF16 weights, and it does include a chat template inherited from LFM3-1.7B, complete with support for “thinking traces” (a reasoning trace prepended to the response) that can be toggled off if you want direct answers instead. Liquid AI also provides Colab notebooks for two fine-tuning paths: supervised fine-tuning for structured tasks like extracting JSON from documents, and continued pre-training for teaching the model a new language from raw text (their example uses Turkish).
How does this fit into Liquid AI’s broader LFM3 lineup?
LFM3-1.7B-Base is one piece of a larger family. Liquid AI’s LFM3 collection includes base, text, vision, audio, omni, and decision models, all presumably sharing architectural lineage even as they target different modalities. The base checkpoint covered here is text-only. Its direct sibling, LFM3-1.7B, had logged over 37,000 downloads and 751 likes on Hugging Face shortly after release, substantially more traction than the base checkpoint itself, which is expected given that most developers want a ready-to-use chat model rather than a raw pre-trained one.
Frequently Asked Questions
What is the difference between LFM3-1.7B-Base and LFM3-1.7B?
LFM3-1.7B-Base is the raw pre-trained checkpoint, trained on 22 trillion tokens but not fully post-trained for chat. LFM3-1.7B takes that same base and adds supervised fine-tuning, mixed reinforcement learning, distillation, and agentic RL, making it ready for chat, tool calling, and agent use cases.
How much memory does LFM3-1.7B-Base need to run?
At 4-bit quantization it needs approximately 1GB of memory, which is small enough to run on phones and standard laptops without dedicated server-grade hardware.
What context length does LFM3-1.7B-Base support?
It supports up to 256,000 tokens of combined input and output, reached through staged training extensions (first to 128K, then to 256K) late in the 22-trillion-token training run.
Can I use LFM3-1.7B-Base directly for chatbots or agents?
Liquid AI doesn’t recommend it for that. It’s intended as a starting point for continued pre-training or custom fine-tuning, such as building domain-specific or language-specific models. For chat and agentic tasks, the post-trained LFM3-1.7B model is the better fit.
What languages does LFM3-1.7B-Base support?
It supports 16 languages: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, and Polish, with a 128,000-token vocabulary.
