How to Run Qwen3.8-27B AEON Uncensored Locally with vLLM
Setup guide for serving the abliterated Qwen3.8-27B AEON model on a single H200 GPU with vLLM, MTP speculative decoding, and tool calling.
What is Qwen3.8-27B AEON and how is it different from stock Qwen?
Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 is an abliterated version of Alibaba’s Qwen/Qwen3.8-27B, released as an Early Access Draft under the AEON-7 name. Abliteration removes the internal refusal behavior trained into the base model, so the checkpoint answers directly instead of hedging, moralizing, or declining. The vision tower and native multi-token-prediction (MTP) head are left unmodified from the base model. It’s distributed in full-precision BF16 and is explicitly described as the master weight set that a future NVFP4 quantized release will be baked from, not a finished product.
TL;DR
- The release is an Early Access Draft, not a general-availability model, and the authors are upfront that long outputs can loop or repeat because the abliteration pass touched some of the same weights that hold coherence together.
- It runs on a single NVIDIA H200 using vLLM 0.27.1 with MTP speculative decoding enabled (3 speculative tokens), and the authors report a draft acceptance rate of roughly 40 to 66 percent during their validation smoke test.
- The model was built by abliterating an SSM-conv1d-repaired copy of Qwen3.8-27B with abliterix 1.12.2, running a 50-trial Optuna search judged by
google/gemini-3.1-flash-lite, and selecting trial 48 of 50 as the coherent point rather than the lowest-KL result. - Refusal-rate testing shows the model dropped from roughly 100 refusals to 36 (29 after smash) on a 100-prompt harmful held-out set, and from 30 to 5 on a 50-prompt sexual-content set, with zero hard “I won’t” refusals across both.
- The measured KL drift (
full_distribution_kl) sits around 0.099 nats/token on harmless held-out prompts, which the authors argue reflects the model no longer spending reasoning tokens on “is this allowed” rather than degraded output quality. - Serving requires
--trust-remote-code, aqwen3reasoning parser, and aqwen3_codertool-call parser, since the model supports tool calling and a thinking mode that’s on by default. - The release ships with an extensive liability and arbitration clause putting full responsibility for prompts, outputs, and downstream use on the operator, since there is no built-in refusal layer left to catch misuse.
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
What hardware do you need to serve this model?
The authors validated the checkpoint on a single NVIDIA H200 GPU running vLLM 0.27.1 on torch 2.13.0+cu130. That’s the tested configuration, not necessarily the floor. Because the model is distributed in BF16 rather than a lower-precision quantization, VRAM headroom matters: the serving command below caps --max-model-len at 16,384 tokens and --gpu-memory-utilization at 0.85 as the validated budget. The model card notes that on a card with roughly 140 GB of memory, you can raise --max-model-len toward the model’s native 262k context window instead of the smaller validation setting. If you’re on a smaller GPU, expect to trade context length and batch size (--max-num-seqs) for headroom, since nothing in the release suggests a quantized (NVFP4 or otherwise) build is available yet. That’s planned as a future sibling release, quantized from this BF16 master, not a substitute for it today.
How do you actually launch it with vLLM?
The model card gives a specific vLLM invocation that enables thinking mode, tool calling, and MTP speculative decoding together:
vllm serve AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 \
--dtype bfloat16 \
--max-model-len 16384 \
--max-num-seqs 4 \
--gpu-memory-utilization 0.85 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--trust-remote-code \
--gdn-prefill-backend triton \
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'A few flags are worth understanding rather than copy-pasting blindly. --reasoning-parser qwen3 and --tool-call-parser qwen3_coder tell vLLM how to parse the model’s thinking blocks and tool-call syntax, both of which are inherited from the Qwen3 family’s chat template. --trust-remote-code is required because Qwen3.8-27B ships custom modeling code. The speculative-config block turns on MTP (multi-token prediction) decoding with 3 speculative tokens per step, using the model’s own native MTP head (grafted back from the stock checkpoint during the build process, since the abliteration merge tool had dropped those 15 tensors). During validation, the authors saw MTP draft acceptance rates of about 40 to 66 percent, which translates into a meaningful throughput gain on supported hardware, since accepted draft tokens skip a full forward pass.
One known snag: if FlashInfer’s sampling JIT can’t find curand.h in your container image, set the environment variable VLLM_USE_FLASHINFER_SAMPLER=0. The model card is explicit that this is an environment and build issue, not a problem with the weights themselves.
How do you control thinking mode and tool calling at inference time?
Thinking is enabled by default on this checkpoint. Per-request, you control it through chat template kwargs, for example:
chat_template_kwargs={"enable_thinking": true, "reasoning_effort": "medium"}This lets you dial reasoning effort up or down per call rather than globally at server startup, which matters if you’re running a mixed workload where some requests need a chain of thought and others don’t. Tool calling works through the standard vLLM auto tool-choice mechanism paired with the qwen3_coder parser flag shown above, so existing OpenAI-compatible tool-calling client code should work with minimal changes once the server is configured correctly.
How was the abliteration actually done?
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
The pipeline documented on the model card goes through several distinct stages. It starts from stock Qwen/Qwen3.8-27B, applies an SSM conv1d outlier repair (a methodology credited to FernflowerAI) to produce an intermediate “ssm-repaired” checkpoint, then runs that through abliterix 1.12.2, a Heretic-derived multi-objective Optuna search tool built for hybrid Mamba/attention architectures. That search ran 50 trials, judged by google/gemini-3.1-flash-lite, scoring thinking output after the </think> tag. The authors picked trial 48 of 50 as the release candidate, explicitly rejecting the lowest-KL trials because those turned out to be no-ops that left refusal behavior fully intact (still near 100/100 on the judge). After export, the MTP head (15 tensors) was grafted back from the stock model via hash-matching, since the abliteration merge process had dropped it, and the vision tower (333/333 tensors) was left completely untouched.
Is the uncensoring reliable, or does it just hide refusals?
The model card is unusually transparent about the limits of automated refusal judging. It draws a distinction between a genuine refusal and a “judge-R”, meaning a case where an automated judge (in this case Flash Lite, the same model used for the smash objective) flags a response as a refusal because it contains a disclaimer, a fictional framing, or a safety resource link, even when the model actually delivered the requested content. Across their 100-prompt harmful held-out set, judge-flagged refusals dropped from about 100 (stock) to 36, and of those 36, the authors say 25 still wrote the requested content in some form (a disclaimer plus payload, a partial draft, or a mislabeled judge call). Across all tested sets, the reported hard refusal count is zero. The tradeoff, and the authors are explicit about this, is that pushing refusal numbers to literal zero on an automated judge is achievable but tends to produce incoherent output, since the same over-editing that kills refusals can also degrade the model’s ability to structure a long answer. That’s the reasoning behind stopping at trial 48 rather than chasing a perfect score.
Frequently Asked Questions
What GPU do I need to run Qwen3.8-27B AEON?
The model card validates the release on a single NVIDIA H200 with vLLM 0.27.1. It’s distributed in BF16, so it needs substantial VRAM headroom; the documented serving command uses a conservative 16k context window and 0.85 GPU memory utilization, with room to extend context length toward the model’s native 262k window on cards with more memory (around 140 GB).
Does this model refuse any requests?
According to the model card’s own testing, hard outright refusals dropped to zero across both a 100-prompt harmful held-out set and a 50-prompt sexual-content set. Some responses are still flagged by automated judges as refusals because they include disclaimers or safety framing, but the authors report that most of those flagged responses still contain the requested content.
Is this the final version of the AEON abliteration?
No. The model card describes this release as an Early Access Draft, not general availability. A later version is planned to be more surgical about preserving coherence on long outputs, and a quantized NVFP4 release is planned to be baked from this BF16 master once that work lands.
What causes the looping or repetition issues mentioned in the model card?
The authors attribute this to the abliteration process editing weights that also support long-form coherence, not just refusal behavior. On typical requests the model behaves like a normal 27B model, but on very long generations, small gaps introduced by the abliteration can compound into repeated phrases or loops.
Does this model support tool calling and vision input?
Yes. Tool calling is supported through vLLM’s auto tool-choice mechanism with the qwen3_coder parser, and the vision tower is inherited unmodified from the base Qwen3.8-27B model, meaning image inputs work the same as they would on the stock checkpoint.