Ripple 65B: Inside Recursive's $32M Open Model Training Recipe
Chip-design lab Recursive trained Ripple 65B, a dense 65B coding model, for $32 million. Here's how the architecture and training actually worked.
What is Ripple 65B and why does it matter?
Ripple 65B is a 65 billion parameter dense language model trained from scratch by Recursive, a lab best known for AI-driven chip design, not language models. Recursive says it built Ripple for about $32 million in total R&D cost and has published an unusually detailed account of how it got there, covering architecture choices, training data volume, and post-training methodology. The model is released under Apache 2.0, with base, instruction-tuned, BFloat16, and NVFP4 versions all available for download.
TL;DR
- Recursive, the company behind the Alpha chip reinforcement-learning research at Google, built Ripple 65B to power its own chip-design platform, then released a general-purpose version publicly.
- Ripple is a dense 65B model, not a mixture-of-experts design, trained on 12 trillion tokens with support for up to a million tokens of context.
- Recursive reports $32 million in total R&D cost, covering experiments, data work, evals, and post-training, not just the final pre-training run, and estimates that’s roughly a tenth of what the previous open-weight leader spent.
- On Terminal Bench 2.1, Ripple scored 80.52, ahead of several much larger US open models, though multiple Chinese open models (DeepSeek, Xiaomi’s MiMo, Kimi, GLM) score higher on the same self-reported benchmark.
- The architecture is thin and deep (80 layers), uses more attention heads than typical, leans on sliding window attention, and borrows recent techniques like gated attention and no positional encoding on global layers.
- Training used laddering, a method pioneered in open form by AI2’s OLMo project, where changes are tested on small models first and only scaled up if they hold.
- Post-training relied entirely on self-built synthetic data and RL environments, reaching its benchmark score with roughly 1.4 million RL rollouts, which Recursive says is about 70 times fewer than a comparable rival used.
Who is Recursive and why did a chip company build an LLM?
Recursive (spelled with an “i”, distinct from Richard Socher’s company of a similar name) is a relatively new lab whose core business is chip design, not language models. Its founders worked on Alpha chip, Google’s reinforcement-learning system for automating chip layout. The company’s broader thesis connects the two fields directly: use AI to design better chips, then use those better chips to train better AI.
That thesis explains why a chip-design company ended up training a 65-billion-parameter language model. Recursive needed an LLM to power its own chip-design platform and wanted full control over the stack, meaning the model itself, the training pipeline, and the serving infrastructure. The fully specialized, chip-design-tuned version of the model stays proprietary. Ripple 65B is the general-purpose version built on the same base, released publicly along with the underlying artifacts.
How does Ripple 65B compare to other open coding models?
On Terminal Bench 2.1, a benchmark for agentic coding tasks, Ripple scored 80.52. That edges out some considerably larger US open-weight models, including a 500-billion-parameter model that scored 80.1 and Meta’s open model at 816 billion parameters, which scored 76.4. Recursive’s own framing of the result is specifically “US open-weight state of the art,” and that qualifier matters.
Several Chinese open models score meaningfully higher on the same benchmark: DeepSeek’s Flash 4.1 reportedly hit 90.6, Xiaomi’s MiMo 2.6 Pro came in at 89.9, and Kimi K3 and GLM 5.3 both landed around 88. Roughly ten Chinese open models currently sit 5 to 10 points above Ripple on this benchmark.
The more useful comparison may be architecture-matched rather than benchmark-ranked. Most of the top-scoring large models, including DeepSeek’s, are mixture-of-experts (MoE) systems with hundreds of billions of total parameters. The closest architectural peer to Ripple is Qwen’s 3.8 27B dense model, also released under Apache 2.0, which scores around 73 on the same benchmark. Ripple is a bit over twice the size and scores about 7.5 points higher.
It’s also worth noting that most of these benchmark numbers, including Ripple’s, are self-reported using different evaluation harnesses. Independent reruns of Terminal Bench 2.1 by third parties have produced notably lower scores for some models than their vendors originally reported, with results dropping into the high 60s for models that claimed scores in the high 80s or 90s. Harness choice and configuration can swing results significantly, so cross-lab benchmark comparisons deserve some skepticism regardless of which lab is reporting them.
The fairest summary: Ripple isn’t the best open coding model available today. It appears to be the strongest small dense model from a US lab that ships open weights, a base model, and a published training recipe, and it’s competitive with Chinese open models at a fraction of the parameter count.
What does the $32 million actually cover?
Recursive is explicit that $32 million represents total R&D cost, not just the compute for the final training run. That figure includes experimentation, data pipeline work, evaluation infrastructure, and the entire post-training process. This is a meaningfully different accounting than headline figures some labs report for just a final RL training pass, which can look dramatically cheaper in isolation.
Recursive estimates this total is roughly a tenth of what it believes the previous open-weight leading model cost, based on publicly reported compute deals. That’s their own estimate and should be treated as directional rather than precise, but it lines up with a broader trend of smaller, well-run teams closing performance gaps with much larger compute budgets through better data and training choices rather than raw scale.
Why did Recursive choose dense over mixture-of-experts?
Nearly every major lab training large models recently has gone the MoE route, since it offers cheaper inference per token by activating only a subset of parameters for any given request. Recursive deliberately went the other direction, choosing a dense architecture, and gave a straightforward reason: dense models are simpler to train and simpler to serve.
For a comparatively small team, that simplicity has real value. MoE architectures require keeping many experts’ weights resident in memory during serving and add real complexity to infrastructure and routing logic. Recursive reported achieving around 40% model FLOPs utilization (MFU) through both pre-training and post-training without relying on custom kernels or quantization tricks, suggesting the dense approach let a smaller engineering team hit solid efficiency without exotic infrastructure work.
The architecture itself is described as thin and deep, with 80 layers and more attention heads than typical for a model this size. To offset the cost of that head count, Recursive leaned on sliding window attention layers, keeping the model smaller and reducing the number of fully global parameters. The design also incorporates several recent but individually non-novel techniques, including gated attention and the removal of positional encoding on global layers.
What is “laddering” and why does it matter for training efficiency?
One of the more notable methodological choices was Recursive’s use of a model ladder, a technique built on work originally published by AI2 (Allen Institute for AI) through its OLMo project. AI2 was, for a long time, one of the only organizations publishing full open training recipes, datasets, and architecture details, though their models tended to stay fairly small due to funding constraints.
A model ladder works by training a series of models at increasing sizes and testing architectural or training changes at the smaller scales first, before committing compute to scaling them up. Recursive used this approach to test changes one at a time, keeping what held up and discarding what didn’t. One example raised was a technique called hyperconnections, which looked promising at small scale but showed no benefit once tested at 4 billion parameters and above, so the team dropped it rather than carrying it into the full-scale run.
This matters because architectural choices that work at small scale frequently fail to transfer to larger models, a problem visible in other model families where small and large variants within the same family use genuinely different architectures. Laddering is a way to catch that mismatch cheaply, before spending the bulk of a training budget on a choice that won’t hold.
Without shadow IT.
How was Ripple’s post-training data built?
Recursive says it did not purchase any post-training data. Instead, the team built its own synthetic data pipelines and RL training environments, progressively adding harder reinforcement learning tasks as the model improved during training. The reported outcome was reaching the model’s 80.52 Terminal Bench score using around 1.4 million RL rollouts, which Recursive characterizes as roughly 70 times fewer rollouts than a comparable large model used to reach its own results.
Recursive also credits prior open work from organizations including AI2, Prime Intellect, and Nvidia as foundational to its approach, rather than presenting the recipe as built entirely in isolation.
Is Ripple 65B worth checking out?
For anyone who needs a model that runs on a single workstation rather than a data center, has a full base model available for custom fine-tuning, ships under a permissive Apache 2.0 license, and comes with an actual published account of how it was trained, Ripple 65B is one of the more interesting releases in that category. It is not the top-scoring open coding model overall, and several larger MoE models from Chinese labs outperform it on the same benchmark. But as a dense, US-trained, fully open alternative with a transparent recipe, it fills a gap that’s become increasingly rare as frontier labs keep both their weights and their methods closed.
Frequently Asked Questions
What is Ripple 65B?
Ripple 65B is a 65 billion parameter dense language model built by Recursive, a chip-design company, trained on 12 trillion tokens with support for up to a million tokens of context. It’s released under Apache 2.0 in base, instruction-tuned, BFloat16, and NVFP4 formats.
How much did Ripple 65B cost to train?
Recursive reports approximately $32 million in total R&D cost, which the company says covers the full process including experimentation, data pipeline development, evaluations, and post-training, not just the final pre-training compute run.
Is Ripple 65B better than Chinese open models?
Not on raw benchmark scores. On Terminal Bench 2.1, Ripple scored 80.52, while several Chinese open models including DeepSeek’s Flash 4.1, Xiaomi’s MiMo 2.6 Pro, Kimi K3, and GLM 5.3 scored higher, roughly 5 to 10 points above Ripple. Recursive’s claim is specifically that it’s the strongest US open-weight model, not the strongest open model overall.
Why did Recursive use a dense architecture instead of mixture-of-experts?
Recursive chose dense over MoE because it’s simpler to train and simpler to serve, requiring less infrastructure complexity for a smaller team, even though MoE architectures generally offer cheaper per-token inference costs at scale.
What is a model ladder in LLM training?
A model ladder is a training methodology, used openly by AI2’s OLMo project and adopted by Recursive for Ripple, where a series of progressively larger models are trained to test architectural and training changes cheaply at small scale before committing full compute budgets to scaling them up.
