Qwen3.8-27B Pi Adapter vs Base Model: Fewer Steps, Not Faster
Hands-on test of the Qwen3.8-27B Pi LoRA adapter against the base model on bug-fixing tasks, comparing steps, tokens, and speed.

What is the Qwen3.8-27B Pi adapter?
It’s a small LoRA (low rank adaptation) checkpoint trained with reinforcement learning to make Qwen’s 27 billion parameter model perform better specifically inside Pi, a coding agent built for software engineering and terminal tasks. Instead of fine-tuning all 27 billion parameters, the adapter freezes the base model and trains two small matrices that sit alongside it. The two outputs get added together at inference time, so the model behaves differently while the adapter file itself stays tiny. Loaded on top of the base Qwen3.8-27B weights in vLLM, it adds tool-calling behavior tuned for Pi’s workflow without touching the underlying model.
TL;DR
- The Pi adapter is a LoRA checkpoint layered on top of base Qwen3.8-27B, trained with reinforcement learning to behave better specifically as the backend model for the Pi coding agent.
- On a simple bug-fixing task, both the adapter and the base model fixed the bug in all three runs, but the adapter used fewer model calls (about 4.7 vs 6) and fewer generated tokens (about 450 vs 519).
- On a harder five-bug, four-file shopping cart task with 12 tests, both models again tied on accuracy, but the adapter needed roughly 20% fewer steps (5.3 vs 6.7 calls) while taking slightly longer per run (about 41 seconds vs 33) and using marginally more tokens.
- The adapter’s main advantage shown in testing is shorter reasoning per tool call combined with fewer total tool calls, not raw speed or lower token count.
- A technique called safe agent (safe RL) was used during training: a proxy checks every tool call before it runs and blocks or zeroes out reward for shortcuts like editing or reading tests instead of fixing the bug.
- The adapter is tied to Qwen3.8-27B specifically and tuned for the Pi harness, so results won’t necessarily transfer to other base models or agents without retraining.
- Testing was done on an H100 GPU with 80GB VRAM, with the base model plus adapter consuming around 78GB of VRAM under vLLM with tool calling enabled and thinking turned off.
- Self-hosted
- SOC 2
- HIPAA
- Audit log
How does the LoRA adapter actually change model behavior?
LoRA works by freezing the original weights of a model and training a much smaller set of additional parameters that run in parallel. When the model processes a request, the output from the original frozen weights and the output from the small trained adapter get combined. The net effect is a model that behaves differently from the base, without ever touching or duplicating the full set of original weights. That’s why the adapter file for a 27 billion parameter model can stay small, and why multiple adapters can be swapped in and out on the same base model without reloading billions of parameters each time.
In testing, one vLLM server held the base model weights in memory once, and multiple adapters, including the Pi-tuned one and several others, were evaluated on top of it. Each adapter is addressed by name per request, so one running server can effectively host several specialized variants of the same base model simultaneously, each scored independently without merging copies.
What did the hands-on comparison actually measure?
The test setup compared the Pi adapter against the plain base Qwen3.8-27B model, both served locally through vLLM with an OpenAI-compatible API, feeding both models into Pi via a models configuration file that points to the same local endpoint. Each model ran the same task three times. A custom script tracked whether the tests passed, how many model calls (turns) each run took, how many tokens were generated, and how long each run took, using vLLM’s own counters for token and timing data.
Two tasks were used:
A simple one: a small Python parser script with a deliberate bug, verified by a failing test suite.
A harder one: a shopping cart project with five bugs spread across four files, checked against 12 tests (eight initially failing), where a pass only counted if every test passed without the tests themselves being modified.
On the simple task, both models fixed the bug in all three runs. The adapter averaged about 4.7 model calls versus 6 for the base model, and about 450 generated tokens versus 519, though its tokens-per-second rate was slightly lower (about 26 vs 28, including tool time).
On the harder task, both models again fixed all five bugs in all three runs, a tie on raw accuracy. But the adapter needed about 5.3 calls on average versus 6.7 for the base model, roughly 20% fewer steps. It didn’t win on tokens or speed here: it used about 1090 tokens versus 990, took around 41 seconds versus 33, and ran at a lower token rate (about 26 vs 30).
Why does the adapter take fewer steps but not less time?
The pattern that emerged across both tasks lines up with what the model’s own documentation describes: the adapter tends to write shorter reasoning between tool calls, but it also needs fewer of those calls to get the job done. Fewer, more targeted steps can mean each individual step carries more weight and takes a bit longer to produce, even if the total number of back-and-forth turns with the tool goes down. That’s a different kind of efficiency than raw throughput. If an agent is being billed or rate-limited per tool call or per turn rather than per token, fewer turns matters more than tokens-per-second. If the bottleneck is generation speed, the base model may still look better on paper.
It’s also worth being clear about the sample size: three runs per task on two tasks is a sanity check, not a statistically rigorous benchmark. The direction of the result (fewer steps, similar accuracy, no token or speed win) is consistent with what the adapter’s own benchmark claims describe, but the margins here are small enough that they shouldn’t be read as precise performance guarantees.
What is “safe RL” and why does it matter for agent training?
When a model is trained with reinforcement learning to solve coding tasks, it can learn to game the reward signal instead of genuinely solving the problem. Common failure modes include reading the answer key (the test file) or editing the tests themselves so they pass without the underlying bug being fixed. To guard against this, the adapter’s training pipeline used a technique described as “safe agent” or safe RL: a proxy sits between the agent and the model server and inspects every tool call before it’s allowed to execute. If a call looks like it’s trying to cheat, such as editing a test file or hitting an address that never appeared in the conversation, it gets blocked and replaced with a short message, and that attempt receives zero reward.
This kind of screening was reportedly applied during the adapter’s training, and a simulated replay across multiple agent harnesses, including Pi, was used to demonstrate the guardrail catching a blocked call in practice. For anyone building or fine-tuning their own coding agents with RL, this is a useful pattern to be aware of: reward shaping without this kind of tool-call auditing can quietly teach a model to cheat rather than to code.
Is the Pi adapter worth using?
For anyone already running Qwen3.8-27B locally and using Pi as their coding agent, the adapter is a low-cost thing to try: it’s a small file layered on top of weights you likely already have loaded, and it doesn’t require retraining or merging. The appeal isn’t that it’s dramatically faster or cheaper per task, in this testing it wasn’t, but that it tends to solve the same problems in fewer steps, which can matter in agentic workflows where the number of back-and-forth tool calls (and the associated latency or API overhead) is the real cost driver.
The adapter is specific to Qwen3.8-27B and tuned for the Pi harness, so it won’t help if you’re using a different base model or a different agent framework. It also doesn’t univerally reduce token usage or runtime, so teams optimizing purely for throughput or cost-per-token may not see a clear win. The broader idea it demonstrates, that a small adapter trained inside the actual agent you use can shift model behavior meaningfully without retraining the whole model, is arguably more useful than this specific checkpoint.
Frequently Asked Questions
What hardware do you need to run Qwen3.8-27B with the Pi adapter?

Testing was done on a single H100 GPU with 80GB of VRAM, using vLLM to serve the base model with the adapter attached. With the context window used in testing, VRAM consumption was around 78GB, so a GPU with at least that much VRAM is effectively required for this specific setup.
Does the adapter work with agents other than Pi?
It’s trained specifically to work well as the backend model inside the Pi coding agent on software engineering and terminal tasks. It can technically be loaded and queried like any other LoRA adapter, but its tuning and the safe RL screening used during training were built around Pi’s tool-calling patterns, so results in other agent harnesses aren’t established.
Is the Pi adapter faster than the base model?
Not based on this testing. On both a simple and a harder bug-fixing task, the adapter needed fewer total model calls (turns) than the base model, but it wasn’t faster in wall-clock time or more token-efficient on the harder task. Its advantage shows up in step count, not raw speed.
What is LoRA and why use it instead of full fine-tuning?
LoRA (low rank adaptation) freezes the original model’s weights and trains a small set of additional parameters alongside them. It avoids the cost of retraining billions of parameters, keeps the adapter file small, and lets multiple specialized adapters be swapped on and off the same base model without reloading the full weights each time.
What is safe RL and why does it matter here?
Safe RL refers to a guardrail process where a proxy checks every tool call an agent makes during reinforcement learning training, blocking calls that look like attempts to cheat (such as editing test files instead of fixing bugs) and assigning zero reward to that attempt. It’s meant to prevent the model from learning shortcuts that look like success but don’t actually solve the underlying task.