on-device LLM Articles
Browse 6 articles about on-device LLM.
LFM2.5-24B-A2B Benchmarks: How It Beats Bigger MoE Models
LFM2.5-24B-A2B edges out larger MoE rivals on agentic and tool-use benchmarks while using far fewer active parameters per token.
Bonsai 2 27B: A 27B Model That Runs in Under 6GB on a Laptop
Bonsai 2 27B uses ternary quantization to shrink a 27B model to under 6GB while keeping 98% of FP16 performance. Here's how it works.
Needle 3 Benchmarks: How a Tiny Model Beats 10x Larger LLMs
Needle 3 packs tool-calling and extraction into an 8-29 MB file. Here's what its benchmarks against much larger models actually show.
Edge0-35B-A3B: How a 35B MoE Model Runs in 3GB of RAM
Edge0-35B-A3B streams MoE experts from SSD to run a 35B-parameter model in under 3GB active memory. Setup, speed, and hardware needs explained.
MiniCPM5-2B: Running OpenBMB's 2B On-Device Model Locally
MiniCPM5-2B packs 2.5B params, 131K context, and 4B-beating benchmarks. Here's how it works and how to run it via GGUF, MLX, or GPTQ.
How to Run Maple-Preview Locally on a Mac Mini M4
A guide to running DeepGrove's Maple-Preview 20B-A1B ternary model on a Mac mini M4, covering hardware needs and real-world speed.