Skip to main content
MindStudio
Pricing
BlogAbout
My Workspace
Inkling 2 Mini benchmarkThinking Machines Lab modelInkling 2 Mini vs Qwen

Inkling 2 Mini Benchmarks: How Thinking Machines' Open Model Scores

Inkling 2 Mini posts strong SWE-Bench and coding scores for a 26B MoE model. Here's how it stacks up against Gemma 4, Qwen, and Nemotron.

Edited by Luis Chavez-Mattos, Director of Product RSS
Inkling 2 Mini Benchmarks: How Thinking Machines' Open Model Scores

What is Inkling 2 Mini and how does it benchmark against other open models?

Inkling 2 Mini is a 26.4B parameter multimodal model from Thinking Machines Lab, released under Apache 2.0 on November 20, 2026, with only 3.4B parameters active per token (around 5.4B effective active parameters once its looped layers are counted). On the model card’s own evaluation table, it leads or ties for the top score on most coding and agentic benchmarks among sub-100B open models, including SWE-Bench Verified (79.2%), SWEBench Pro v2 Hard (77.9%), and SWEBench Multilingual (73.7%), outperforming Gemma 4 31B, Qwen3.8-27B, and Nemotron 3.5 Super VL on most of these tests.

TL;DR

  • Inkling 2 Mini is a 26.4B total parameter, 3.4B active parameter mixture-of-experts model built by Thinking Machines Lab and released open weight under Apache 2.0.
  • On coding benchmarks, it posts the best or second-best score in the comparison table on FrontierCode v2.0 (15.1%), DeepSWE v1.1 (48.2%), SWEBench Pro v2 Hard (77.9%), SWEBench Multilingual (73.7%), and SWEBench Verified (79.2%).
  • Its closest rival across most of these tests is Qwen3.8-27B, which beats Inkling 2 Mini on Terminal Bench v2.1 (73.0% vs 71.6%) but loses on nearly every other coding metric listed.
  • The architecture uses a looped transformer design: 24 layers of weights, but the middle 12 layers run three times, giving each token 48 effective layer passes while keeping the stored weight count small.
  • At inference, a real-world agentic test (debugging a tiered electricity pricing bug in a solar panel app) caused the model to loop for over an hour without finding the root cause, a notable gap between benchmark scores and live agentic performance.
  • The model handled vision and reasoning tasks well in hands-on testing: it correctly rebuilt a hand-drawn sketch into an interactive HTML simulation and avoided fabricating details about a fictional photo, though it showed weak recall on a harder flag-identification task.
  • It is small enough to run locally on 32GB or more of VRAM or unified memory (DGX Spark, Apple silicon with 32GB+), with a 20GB checkpoint in mixed NVFP4/MXFP8 precision and support for a 1M token context window.
✗ VIBE-CODED APP
Tangled. Half-built. Brittle.
✓ AN APP, MANAGED BY REMY
UIReact + Tailwind✓
APIValidated routes✓
DBPostgres + auth✓
DEPLOYProduction-ready✓
Architected. End to end.

Built like a system. Not vibe-coded.

Remy manages the project — every layer architected, not stitched together at the last second.

What’s inside the Inkling 2 Mini architecture?

Inkling 2 Mini is described in its model card as a “multimodal autoregressive looped transformer.” Instead of the standard setup where data passes through each layer exactly once, Inkling 2 Mini has 24 layers of weights, but the middle 12 run three times in sequence, reusing the same weights each pass. That gives every token 48 layer applications total despite only 24 layers of weights stored on disk. Each loop pass starts from the output of the previous one, with a gating mechanism controlling how much new information gets blended in versus carried forward, so repeated passes refine rather than just repeat the same computation.

Inside each layer, the model uses sparse mixture-of-experts routing: 128 expert blocks exist, but each token is routed to only 8 of them. Combined with the looping, this is how a 26.4B parameter model ends up with roughly 5.4B effective active parameters per token once all three passes are counted. Attention is a hybrid of local layers, which mostly attend to the last 1,024 tokens, and global layers (every sixth layer) that attend to the full context, which is how the model supports a 1 million token context window without every layer paying full quadratic attention cost.

The tradeoff, as observed in independent testing, is that the looping and per-pass memory add real inference latency. The model is smaller on disk than its parameter count suggests, but each forward pass does more work, which can make response times slower than the raw parameter count would imply.

How does Inkling 2 Mini perform on coding benchmarks?

According to the official model card evaluation table, Inkling 2 Mini scores at or near the top of its weight class across several coding evaluations, compared against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL:

  • FrontierCode v2.0: 15.1% (best), just ahead of Qwen3.8-27B at 14.9%, with Gemma 4 and Muse Glimmer near zero.
  • DeepSWE v1.1: 48.2% (best), versus 42.2% for Qwen3.8-27B and 24.7% for Nemotron.
  • Terminal Bench v4.0: 5.6%, tied for best with Qwen3.8-27B, while Gemma 4 and Nemotron score near 0%.
  • Terminal Bench v2.1: 71.6%, second-best behind Qwen3.8-27B’s 73.0%.
  • SWEBench Pro v2 (Hard): 77.9% (best), ahead of Qwen3.8-27B (73.8%) and well ahead of Gemma 4 (14.2%) and Nemotron (44.1%).
  • SWEBench Multilingual: 73.7% (best), ahead of Qwen3.8-27B (70.0%) and Nemotron (67.7%).
  • SWEBench Verified (bash only): 79.2% (best), ahead of Muse Glimmer at 76.0%.

The pattern across these numbers is consistent: Qwen3.8-27B is the only model in the comparison that regularly comes close to Inkling 2 Mini, and it wins outright on Terminal Bench v2.1. Gemma 4 31B lags well behind on every coding benchmark in the table, and Nemotron 3.5 Super VL, despite having a much larger total parameter count (125B total, 12B active), trails both Inkling 2 Mini and Qwen on most coding metrics.

On broader indices, Inkling 2 Mini ties Qwen3.8-27B for the top AA Index score (20.1) and leads the Vals Index at 46.5%, well ahead of Nemotron’s 29.8%.

Build enterprise applications with AI that are
Secure.Role: Adjuster✕ Export PHI
Regulation-compliant.✓ HIPAA✓ SOC 2
Auditable.09:14M. Chen approved claim #4471
Scalable.12,480 users online

Does strong benchmark performance translate to real agentic coding tasks?

Not always, based on hands-on testing covered in independent review. One tester ran Inkling 2 Mini through a Hermes agent harness against a deliberately planted bug in a realistic solar panel installer app: a tiered electricity pricing calculation that incorrectly charged all usage at the highest price bracket instead of applying each tier’s rate only to the units within that tier. The task required finding the bug from symptoms alone, without hints, then fixing and rebuilding the app.

Inkling 2 Mini ran for over an hour without pinpointing the root cause, appearing to loop and burn through tokens without converging, even after the tester adjusted thinking settings. For comparison, the same reviewer noted that other models, including Qwen3.8, solved similar classes of bugs in five to ten minutes. The test was called a fail and stopped early due to time constraints.

This gap matters for anyone evaluating the model for coding-assistant use cases. Benchmark scores like SWE-Bench Verified are measured in controlled harness conditions with specific agent scaffolding, and the model’s own card lists Terminal Bench and SWEBench results measured with tools like mini-swe-agent and specific harness versions. Real-world agent loops, different system prompts, or different tool-calling setups can produce meaningfully different results than the benchmark table suggests, particularly for an architecture that relies on multiple loop passes and can spend a lot of compute “thinking” before acting.

Is Inkling 2 Mini worth running locally?

For teams with 32GB or more of VRAM or unified memory, Inkling 2 Mini is a realistic candidate to self-host. The checkpoint is about 20GB in mixed NVFP4 and MXFP8 precision, and Thinking Machines Lab lists support for NVIDIA DGX Spark, Apple silicon devices with 32GB or more of memory, and other GPUs with similar VRAM. It’s compatible with common local inference stacks including Hugging Face, SGLang, vLLM, Unsloth (with quantized GGUF versions), llama.cpp, and Ollama.

Outside of coding agent loops, independent testing found the model performed well on vision and reasoning tasks: it accurately converted a hand-drawn sketch (including handwritten labels) into a working interactive HTML simulation, correctly traced a multi-step flow diagram and explained it accurately, and refused to fabricate details about a photo falsely claimed to be from a nonexistent movie, correctly identifying what it could and couldn’t verify. It struggled on a harder visual discrimination task involving near-identical national flags, and the model card itself states it is not strongly multilingual and performs best in English.

For developers who want a small, open, image-and-text capable model with a 1M token context window and genuinely competitive coding benchmark scores, Inkling 2 Mini is a reasonable pick. For anyone relying heavily on long autonomous agentic coding sessions, the gap between its benchmark numbers and at least one real-world agent test is worth factoring in before committing it as a primary coding assistant.

Frequently Asked Questions

What is Inkling 2 Mini?

The enterprise platform for building software with AI

Build an onboarding app for 4,000 new hires.

Build ↵
Self-hosted
SOC 2
Role-based access
Audit log

Inkling 2 Mini is an open-weight multimodal model from Thinking Machines Lab, released under Apache 2.0 on November 20, 2026. It accepts text and image input and produces text output, and uses a looped transformer architecture with mixture-of-experts layers, totaling 26.4B parameters with 3.4B active per forward pass (about 5.4B effective active parameters counting its loop structure).

How does Inkling 2 Mini compare to Qwen3.8-27B?

The two models are close competitors. Inkling 2 Mini leads on most coding benchmarks in its own evaluation table, including SWEBench Pro v2 Hard, SWEBench Multilingual, SWEBench Verified, and DeepSWE v1.1. Qwen3.8-27B wins on Terminal Bench v2.1 and ties Inkling 2 Mini on Terminal Bench v4.0 and the AA Index.

Can I run Inkling 2 Mini on a regular laptop?

It requires at least 32GB of VRAM or unified memory, so it runs on Apple silicon MacBooks, Mac minis, and Studios with 32GB or more of memory, or on hardware like the NVIDIA DGX Spark. It won’t run comfortably on lower-memory consumer laptops without aggressive quantization.

Why did Inkling 2 Mini fail a real-world coding agent test despite strong benchmarks?

In one independent test, the model looped for over an hour trying to debug a tiered pricing calculation error without reaching a fix, despite scoring well on standardized coding benchmarks. This points to a difference between benchmark harness conditions and open-ended agentic debugging, where the model’s looped architecture may spend excessive compute without converging on a solution.

Is Inkling 2 Mini good at multilingual tasks?

No. The model card states the model is not strongly multilingual and performs best in English, so developers needing reliable non-English performance should test other models.

Editorial standards

Related Articles

Inkling 2 Mini: Run Thinking Machines' Open Model Locally

Inkling 2 Mini is a 26.4B-parameter open-weight multimodal model from Thinking Machines Lab, runnable locally on a 32GB Mac or DGX Spark.

Inkling 2 MiniThinking Machines Lab modelrun Inkling 2 Mini locally

HP ZGX Fury Benchmarked: Local LLM Speeds for DeepSeek, GLM, Qwen

HP ZGX Fury benchmarks for local inference: token speeds, concurrency, and VRAM math running DeepSeek, GLM 5.3 Flash, and Qwen models.

HP ZGX Furylocal LLM benchmarksNVFP4 quantization

Recursive Flint's SWE-bench and Terminal-Bench Scores, Explained

Recursive Flint hits 89.7% on SWE-bench Pro v2 and 54.9% on Terminal-Bench 2.1. Here's what those numbers mean for a 40B open coding model.

Recursive Flint benchmarkSWE-bench VerifiedTerminal-Bench

Naive-N0.5-Flash: Specs and Setup for the 309B MoE Coding Model

How to run Naive-N0.5-Flash locally: hardware needs, FP8 quantization, context window, and setup steps for this 309B MoE coding model.

Naive-N0.5-Flashrun Naive-N0.5-Flash locallyopen weight coding model

GLM-5.3: ZAI's New Model Shows Unexpected Cybersecurity Skills

ZAI's GLM-5.3 pairs frontier coding benchmarks with surprising cybersecurity gains, arriving first through a coding plan ahead of open weights.

GLM-5.3ZAIcybersecurity AI model

Hi-4 Preview Hands-On: Coding, 3D Games, and Document Audits

Hands-on tests of Tencent's Hi-4 Preview model tackling a platformer, a 3D racing game, and a 24-claim expense audit in one shot.

Hi-4 preview demoWorkBuddy AIAI coding test

Presented by MindStudio

No spam. Unsubscribe anytime.