NeoHorse-1-4B: How to Run This Self-Improving 4B Model Locally
NeoHorse-1-4B trains on router decision logs to improve itself. Here's how to install and run this 4B open model locally with vLLM.
What is NeoHorse-1-4B?
NeoHorse-1-4B is a 4 billion parameter open model from TokenRhythm, built on top of Qwen3.5-4B and released under an Apache 2.0 license. It’s not the architecture that makes it notable. It’s the training method: instead of learning from static datasets alone, NeoHorse was trained on logs from a live routing system, the kind of infrastructure that decides which AI model handles which incoming task. Those decision logs, ordered from easy to hard and refined with a stronger teacher model correcting live attempts, became the fine-tuning data. The idea is that the model learns from real operational patterns rather than curated examples written in advance.
TL;DR
- NeoHorse-1-4B is a Qwen3.5-4B finetune from TokenRhythm trained on router decision logs instead of static instruction data.
- The training approach uses recursive self-improvement: a router’s real task-routing decisions become training data, refined by a teacher model, and each improved version can generate new logs for the next round.
- It runs locally through vLLM with modest VRAM requirements typical of a 4B dense model, making it practical on a single consumer or prosumer GPU.
- In a hands-on agentic coding test, the model found and fixed a subtle “false all clear” bug in a multi-service dashboard app without being told where the bug was.
- On a creative writing test, it produced fluent, stylistically strong prose but lost consistency in details like character relationships and setting.
- Benchmark charts comparing it against five similarly sized models show it ranking at or near the top across agentic tasks, coding, and instruction-following, with a strong average across ten benchmarks.
- The self-improvement loop currently happens during training, not inference, so the model doesn’t rewrite itself while answering your prompts.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
How does NeoHorse-1-4B’s self-improvement training actually work?
The core mechanism is a feedback loop built around a model router. In production AI systems, a router sits in front of several models and decides, for each incoming request, which model should handle it. That routing decision, along with whether the chosen model actually succeeded, normally just sits in a log file as operational data.
TokenRhythm’s approach turns that log into training signal. The interaction data gets ordered from easier tasks to harder ones, so the model builds up capability gradually rather than getting thrown at difficult problems immediately. A second phase adds a teacher model that watches NeoHorse attempt tasks in real time and corrects it in the moment, rather than just supplying pre-written “correct” answers after the fact.
The recursive part comes after deployment: once an improved version goes back into service, it generates new interaction logs of its own performance, which then become training data for the next iteration. It’s a loop where the system’s own usage data is the fuel for the next round of training. As of now, this recursion happens during training cycles between model versions, not as the model runs. There’s no live self-rewriting happening mid-conversation.
How do you install and run NeoHorse-1-4B locally?
The model is distributed on Hugging Face (TokenRhythm/NeoHorse-1-4B) in safetensors format, split across two shards, with a standard config.json, tokenizer files, and chat template. It’s tagged as a qwen3_5_text architecture, so anything that already supports Qwen3.5 should support NeoHorse with minimal changes.
The most common way to serve it locally is vLLM. A basic flow looks like:
- Install vLLM in a Python environment with a compatible CUDA toolkit.
- Download or point vLLM at the
TokenRhythm/NeoHorse-1-4Brepo on Hugging Face. - Launch an OpenAI-compatible server with vLLM’s serve command, specifying the model path and any context length or quantization flags you need.
- Point your client, coding agent, or chat frontend at the local endpoint vLLM exposes.
Because it’s a 4B dense model, VRAM overhead is modest by current standards. Actual consumption depends on context length and KV cache settings, since longer contexts and larger batch sizes eat into the KV cache budget on top of the base weights. For a model this size, it’s realistic to run on a single consumer GPU with room to spare for reasoning-heavy, longer-context workloads. If you don’t have a suitable local GPU, renting one from a cloud GPU provider is a common workaround for testing before committing to local hardware.
Is NeoHorse-1-4B actually good at coding tasks?
In one hands-on test, NeoHorse was pointed at a running full-stack demo app, a glacial flood early-warning dashboard with a Docker frontend and backend and a live database, and given a single instruction: find and fix every bug. No hints about what was broken or where.
The app had a subtle but serious flaw: it displayed real-time flood advisories and simulated a glacier breach correctly in the scrolling detail view, but the summary header at the top kept showing “zero people in inundation zone” even after the breach happened, a false all-clear that’s arguably worse than an app that just crashes.
The model, connected through an agent framework, worked through the codebase with visibly heavy reasoning, checking and rejecting its own hypotheses across multiple endpoints before converging on a fix. After it finished, reloading the app showed the corrected numbers (over 41,000 people in the inundation zone) and the affected towns properly flagged. For a 4B parameter model, that’s a nontrivial agentic debugging task handled without being told where to look.
Is NeoHorse-1-4B good at creative writing?
Less consistently. In a test asking the model to write a non-fiction-style piece about an immigrant living alone, built entirely around a week’s worth of WhatsApp messages (spam, bills, junk), the model produced prose that was structurally strong and stylistically confident, with literary techniques like fragmented sentences, white space, and ellipses used well.
The problem was consistency. The piece never settled on one country, one language, or one stable family structure. Characters described as mother, son, daughter, and grandchild blurred into what read like the same person across different passages. That’s a coherence failure more than a prose-quality failure, and it suggests the router-log training approach, which seems to pay off heavily on agentic and coding benchmarks, doesn’t automatically transfer to long-form creative consistency.
How does it compare to other 4B models?
TokenRhythm’s own benchmark comparisons place NeoHorse against five other models in the same weight class (roughly 4B parameters) across ten benchmark categories covering agentic tasks, coding, and instruction-following. NeoHorse lands at or near the top in most categories, and its average score across all ten benchmarks is competitive with or ahead of the comparison group. The pattern isn’t a model that dominates one narrow skill, it’s one that’s consistently solid across categories, which lines up with the hands-on coding test but only partially with the creative writing result.
Frequently Asked Questions
What base model is NeoHorse-1-4B built on?
It’s a finetune of Qwen3.5-4B, using the same qwen3_5_text architecture family, so tooling built for Qwen3.5 models generally works with it out of the box.
Do I need a powerful GPU to run NeoHorse-1-4B?
No. At 4 billion parameters, it’s small enough to run on a single consumer or prosumer GPU using vLLM, with VRAM use scaling mainly with context length and KV cache settings rather than the base model weights alone.
Does the model actually improve itself while running?
Not during inference. The “self-improving” part refers to a training pipeline where router logs and teacher-model corrections feed into each new training round, and that improved model then generates fresh logs for the next iteration. It’s recursive across model versions, not within a single running session.
Is NeoHorse-1-4B better at coding or creative writing?
Hands-on testing found it notably strong at agentic coding and debugging tasks, including finding and fixing a bug without being told what or where it was. Creative writing output was fluent and stylistically solid but struggled with consistency across characters, settings, and language choices.
Where can I download NeoHorse-1-4B?
It’s hosted on Hugging Face under TokenRhythm/NeoHorse-1-4B, released with an Apache 2.0 license, with safetensors weights and standard tokenizer and config files included.
