Vast-10M Hands-On: Does Waltropy's 10M Context Claim Hold Up?
Hands-on testing of Waltropy's Vast-10M and its VSA attention mechanism, which claims 10 million tokens of native context.
What is Vast-10M and what does it claim?
Vast-10M is a model family from Waltropy built by replacing the attention mechanism in three existing open models (Deepseek V4 Flash, Deepseek V4 Pro, and GLM 5.2) with a custom mechanism called VSA, short for Waltropy Scalable Attention. The result ships in flash and medium variants, each advertised with 10 million tokens of native context. Waltropy’s own technical report shows the model beating Anthropic’s Claude at 1 million tokens of context on its benchmark chart. That’s a strong claim, and the only way to know if it means anything in practice is to run it against real agentic and long-document tasks.
TL;DR
- Vast-10M swaps attention, not architecture: Waltropy didn’t train a new foundation model from scratch. It took Deepseek V4 Flash, Deepseek V4 Pro, and GLM 5.2 and replaced their attention layers with VSA, claiming this extends context to 10 million tokens without heavy retraining.
- Long agentic coding tasks expose real gaps: a complete 3D platformer build (Blender assets plus Godot gameplay code) took roughly an hour to generate and produced a game that launched but was missing the player character logic, so jumping and movement worked but nothing visible moved.
- Single-file creative generation was a partial win: a detailed HTML biker tattoo prompt produced a working, error-free page with several requested elements (skull, roses, flame cluster, mandala, banner), though finer details like the snake scales and winged crown were missing or simplified.
- Legal document retrieval across millions of tokens was the strongest result: tested against seven volumes (around 4MB of text) of Australia’s Corporations Act, the model gave a well-structured answer, correctly flagged sections it couldn’t verify instead of guessing, and a spot-checked citation matched the real law.
- The model’s internal reasoning looped and second-guessed itself: in the legal test, the reasoning trace showed repeated rechecking of which files were loaded and some mixed-up volume names, even though the final answer was accurate.
- Multilingual creative output was mostly solid but overclaimed: an 81-language dating-app-name generation task returned a clean, complete table and honestly flagged two uncertain languages, but the model’s claim that no two names shared the same root was false, since at least nine entries clearly repeated the “love” or “longing” concept across languages.
- Documentation on VSA itself is thin: there’s limited public detail on how the mechanism works internally, and Waltropy has not open-sourced or open-weighted the model, which limits independent verification of the context-extension claims.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
How does VSA actually extend context to 10 million tokens?
VSA (Waltropy Scalable Attention) is positioned as a drop-in replacement for the standard attention mechanisms inside existing transformer models. Rather than building a new model from the ground up, Waltropy took three already-capable open models and modified how they attend to tokens, with the stated goal of stretching usable context out to 10 million tokens while also improving scores at shorter context lengths, all without extensive retraining.
The technical report provides only a few paragraphs on the actual mechanics of VSA. There’s no deep architectural breakdown of how it manages memory, computation cost, or accuracy at scale, which makes it hard to independently verify why it should outperform standard attention at long range. For a company making a claim this large (10 million tokens is an order of magnitude beyond what most production long-context models support), more transparency, ideally in the form of open weights or a fuller technical writeup, would go a long way toward building trust in the number.
Can it actually finish a long agentic coding task?
This is where the claims met the most resistance. The test was a complete 3D “floor is lava” style platformer: Blender scripts for furniture assets, Godot 4 code for gameplay, plus a timer, start screen, and game-over screen, all generated and assembled autonomously in one long run.
The model worked through the task for close to an hour and produced a large set of files without crashing or needing intervention mid-run, which is itself a reasonable signal for long-horizon agentic stability. But when the game was launched, the core gameplay was broken. Jumping registered. Movement registered. But there was no visible character on screen, meaning the platformer mechanics existed in code but the player model wasn’t properly wired into the scene. For a task meant to demonstrate end-to-end planning and execution over an extended agentic session, producing a large volume of files without a functioning core loop is a meaningful miss, especially given the time invested.
Is it good at detailed single-shot creative work?
The second test asked for an intricate biker tattoo design on a realistic forearm, rendered as a single self-contained HTML file; essentially a stress test of detailed visual reasoning without an image model in the loop.
The output worked cleanly with no runtime errors, and several requested elements showed up: a skull, crossed pistons, three roses, a red flame cluster, a mandala, and a “ride or die” banner on a dark background. The roses were the most polished element, and the skin tone shading gave a believable sense of depth for a plain HTML/CSS composition. Where it fell short was on the specific, intricate details requested in the prompt: the skull lacked the winged crown called for, the snake looked like a rough black coil rather than having visible scales, and the banner text overlapped slightly. Call it a reasonable first pass that captured the broad composition but missed the fine detail a “museum quality” bar would require.
How well does it handle long-document retrieval?
This is where Vast-10M’s long-context design showed the clearest payoff. The test loaded seven volumes, around 4MB of text, of Australia’s Corporations Act into the model’s hosted platform and asked a legal question that required connecting information across sections scattered throughout the entire document, a classic needle-in-a-haystack scenario but at real document scale rather than synthetic filler text.
The final answer was well structured, followed the grouping requested in the prompt, and explicitly flagged which parts it couldn’t confirm rather than guessing or inventing a citation. A spot check of one cited section against the real statute matched. That combination (accurate on verification, honest about uncertainty) is exactly what you want from a model handling legal or compliance-grade documents at scale.
The catch is in the reasoning trace behind that answer. Watching the model’s intermediate steps showed it looping: repeatedly rechecking which files were loaded, mixing up volume numbers at one point, and at one stage reconstructing a passage from memory and then asserting it was present in the text without actually verifying it first. The final output was solid, but the path to it was messier and less efficient than the clean final answer suggests.
Does it hold up on multilingual and creative tasks?
The last test asked for a culturally specific dating app name in 81 different languages, with a rule against repeating root concepts, as a way to probe linguistic range and honesty about knowledge gaps. The model returned a complete, cleanly formatted table covering all 81 languages with no extraneous text, and it proactively flagged two languages (Gutish and Tigrinya) where it wasn’t confident in the output, which is a good sign for a model you’d trust with low-resource language work.
The problem is the model’s own closing claim that no two entries shared a root concept. That was false. At least nine of the generated names across languages like Pashto, Latin, and Esperanto clearly mapped back to the same idea of “love,” and the concept of “longing” repeated across several other entries too. The generation quality itself was good; the self-assessment of its own output was not accurate.
Is Vast-10M worth using right now?
Based on these tests, Vast-10M is strongest at long-document retrieval and honest about its own limits in that context, which is arguably the most commercially useful property of a 10-million-token context window: you want a model that says “I couldn’t find this” rather than fabricating a citation. It’s weaker on long-horizon autonomous coding, where an hour of agentic work produced a lot of output but not a working core feature, and it’s inconsistent on self-assessment, overclaiming accuracy on its own multilingual creativity. Given that the model isn’t open-weighted and the VSA mechanism isn’t fully documented publicly, treat the 10-million-token headline as promising but not yet independently verifiable at the level of detail serious technical teams would want before depending on it for production workloads.
Frequently Asked Questions
What is VSA in Vast-10M?
VSA stands for Waltropy Scalable Attention, a custom attention mechanism Waltropy built to replace the standard attention layers in existing open models, with the stated goal of extending usable context to 10 million tokens without full retraining.
Which base models does Vast-10M use?
Vast-10M is built on top of three existing open models: Deepseek V4 Flash, Deepseek V4 Pro, and GLM 5.2, each modified with Waltropy’s VSA attention mechanism rather than trained from scratch.
Is Vast-10M open source?
No. As of testing, Waltropy has not released open weights or a fully open technical report for Vast-10M, which limits independent verification of its architecture and long-context claims.
How did Vast-10M perform on long legal documents?
In a test against roughly 4MB of text from Australia’s Corporations Act, the model gave a well-structured, accurate answer and correctly flagged information it couldn’t verify, though its internal reasoning process showed some looping and file-tracking errors along the way.
Did Vast-10M complete the agentic coding test successfully?
Not fully. In a roughly one-hour autonomous coding run to build a 3D platformer game, the model generated a large number of files, but the resulting game launched with broken core mechanics, since player movement registered but no character was visibly rendered on screen.
