Mistral Large 4 Coding Test: Can "Le Chonk" Finish What It Starts?
Mistral Large 4 (Le Chonk) faced eight agentic coding tasks in Open Code. Here's how it scored, where it failed, and where it impressed.
What is Mistral Large 4 (“Le Chonk”)?
Mistral Large 4, nicknamed “Le Chonk,” is Mistral AI’s large flagship model, released as a preview and tested here through Open Code via Open Router. In a hands-on evaluation using eight agentic coding tasks, it scored 42 out of 80 points (5.25 out of 10), with results ranging from a near-perfect browser simulation to two complete failures where the model stopped without producing any output at all.
TL;DR
- Mistral Large 4 scored 42/80 (5.25/10) across eight agentic coding tasks run in Open Code through Open Router, using the same KingBench 3 prompt set applied to other models on the channel.
- Reliability, not raw capability, was the main weakness. The model produced zero-point results on two tasks (a 3D contact lens case and a counting problem) simply by ending its turn without delivering a file or an answer.
- When it finished a focused visual build, the output was genuinely strong, with the three-elevator simulation scoring 9.5/10 and the folding table scoring 8.5/10, both rendering and interacting correctly in a real browser.
- The most ambitious task, a local Gemma 2B fine-tune paired with a Panda-facts web page, scored a full 10/10, including a real validation loss drop from about 7.97 to 0.31 over 400 LoRA training steps.
- A broken 3D wristwatch build only became usable after a manual repair pass, where the tester fed the browser error back to the model, and even then it scored just 6/10 due to layout problems.
- An initial zero score on the contact lens task was partly a tooling artifact, since a rerun with reasoning disabled in Open Code produced a working result, though the default run still counted as a failure for reliability purposes.
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
How was Mistral Large 4 tested?
The evaluation used eight fixed prompts, called KingBench 3, previously applied to other models on the same channel for consistency. Each task was worth 10 points, for a maximum score of 80. The model connected to Open Code, an agentic coding interface, through Open Router, and every task started in a fresh folder to avoid any carryover between runs.
The grading rule was strict: only the delivered result counted, not the model’s explanation of what it intended to build. For browser-based tasks, that meant actually opening the output in Chromium, checking whether the page rendered, and testing whether the core interaction worked. This distinction matters because an agent can produce a confident closing summary while the underlying file is broken, incomplete, or never created.
Where did Mistral Large 4 succeed?
The three best results were visual, self-contained builds:
- Elevator simulation (9.5/10): Three animated elevator cars, a spawn button, and a people counter. Passengers queued correctly, destinations showed on hover, and the waiting/riding counters returned to zero with no browser errors across multiple test runs. Minor issues included a leftover animation artifact on reset and a counter that updated slightly before a passenger physically boarded.
- Folding table (8.5/10): A Three.js scene with lighting, shadows, orbit controls, and a slider that smoothly folded and unfolded a 3D table. Both extremes of the slider rendered without errors, though the table’s feet lifted oddly off the floor during the animation.
- Panda SVG (8/10): A clean, recognizable vector illustration of a panda holding a burger. The only weakness was that the burger sat against the mouth without a convincing bite, leaving the “eating” part of the prompt only partly satisfied.
- Local Gemma fine-tune (10/10): The most complex task asked the model to generate a dataset of panda facts, fine-tune a local Gemma 2B model, and serve a web page that displayed a new fact on every refresh. Mistral Large 4 built a 510-example dataset, trained a LoRA adapter for 400 steps, dropped validation loss from roughly 7.97 to 0.31, and even diagnosed and fixed a server threading bug during its own test pass. Eight separate page refreshes returned eight different facts with no errors.
Where did it fail, and why does that matter more than the wins?
Two tasks ended with nothing to grade:
- 3D contact lens case (0/10 on default run): The model reasoned about the design but never created the HTML file. There was nothing to open or click. A separate run with Open Code’s “no reasoning” setting did produce a working file, with independently clickable left and right lids, though the lids flew too far upward on open. That confirms the model can do the task, but the default configuration failed to deliver it, and the default run is what counted toward the score.
- Counting problem (0/10): The correct answer was 20,460. Instead of producing that number or a checkable method, the model again used its turn without delivering a final answer.
A third task, the bow and arrow game, technically ran but fell short on usability: four target types, wind physics, hit tracking, and a saved leaderboard all existed, but targets were only a few pixels wide on a normal desktop screen, were fixed in position (so some could fall off-screen on mobile), and the aiming guide didn’t match the actual arrow trajectory. That scored 6/10, work clearly happened, but the result wasn’t enjoyable to actually use.
The 3D wristwatch task showed the clearest case of a fragile agentic process. The model wrote an ambitious watch page and downloaded Three.js modules to serve locally, but the first browser check showed a blank page and placeholder time displays. The cause was an import map pointing to module paths the model never created. After the exact browser error was fed back into Open Code, the model repaired the paths and produced a working 3D clock with a sweeping second hand and a switchable second time zone. Even then, it only reached 6/10, since the wrist and bracelet dominated the frame and the time zone label didn’t update immediately.
Is Mistral Large 4 reliable enough for agentic coding?
Based on this test, reliability is the open question, not raw capability. When Mistral Large 4 completed a task, the quality was often genuinely good: the elevator simulation and folding table were functional and polished, and the local fine-tuning task showed it can execute a long, multi-step workflow (dataset generation, training, debugging, and serving a page) when the environment is configured correctly.
But two tasks ended with the model simply stopping mid-process, producing no file and no answer, and a third required a manual repair cycle before it worked. For a developer trying to hand off a task and walk away, that’s the practical risk: you don’t always get a clear failure message, you sometimes just get silence where a deliverable should be.
How should developers interpret these scores?
A score of 5.25/10 across eight tasks shouldn’t be read as a universal ranking against other models, since this was one run, on one set of prompts, through one specific Open Code and Open Router configuration. The permission issue that blocked the fine-tuning task on the first attempt, for instance, was a setup problem, not a model failure, and was corrected before that score counted.
The more useful signal is the pattern: Mistral Large 4 does well on contained, single-file visual builds where the scope is clear, and it struggles on tasks that require either a long reasoning chain before output (the contact case, the counting problem) or precise coordination across multiple files and import paths (the watch). Anyone testing it for real work should keep a browser and a manual test loop close by, especially for anything beyond a single self-contained page.
Frequently Asked Questions
What does “Le Chonk” mean in relation to Mistral Large 4?
It’s a nickname used informally for Mistral Large 4, referenced throughout this hands-on test as shorthand for the model.
What tools were used to test Mistral Large 4’s coding ability?
The model was connected to Open Code, an agentic coding interface, through Open Router, and tested against eight fixed KingBench 3 prompts previously used on other models for comparison.
What was Mistral Large 4’s overall score in this test?
It scored 42 out of 80 possible points across eight tasks, equal to 5.25 out of 10 or 52.5%.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
Did Mistral Large 4 fail any tasks completely?
Yes. On a 3D contact lens case and a counting problem, the model ended its turn without producing a usable file or answer, resulting in zero points on both in the default configuration.
What was Mistral Large 4’s best result in this test?
A full 10/10 on a task requiring a generated dataset, a local LoRA fine-tune of Gemma 2B, and a working web page that served a new fact on each refresh, all completed end to end with a measured drop in validation loss.
