Microsoft Aion 1.0 Instruct: Run Windows' Future SLM Locally Today
Microsoft's 2.3B-parameter Aion 1.0 Instruct model previews Windows' on-device AI. Here are its specs, benchmarks, and how to run it now.

What is Microsoft Aion 1.0 Instruct?
Aion 1.0 Instruct is a 2.3 billion parameter small language model (SLM) that Microsoft’s Windows Applied Sciences Group released as open weights on Hugging Face. It’s the successor to Phi Silica, the model currently handling on-device AI tasks in Windows, and a further optimized version of it is slated to become the default Windows OS SLM by the end of January 2027. The Hugging Face release lets developers test the model today, months ahead of its system-level rollout.
TL;DR
- Aion 1.0 Instruct is a 2.3B-parameter causal language model built specifically for on-device, OS-level tasks like summarization, rewriting, and chat rather than broad world-knowledge queries.
- The model supports a 32,768-token context window, enough to process multi-page documents or long email threads entirely on a local device without sending data to the cloud.
- It uses a hybrid attention design with 21 sliding-window layers and 7 full-attention layers, paired with grouped-query attention, to keep memory and compute costs low for edge hardware.
- Benchmark scores show solid instruction-following (83.90 on IFEval) but weaker world knowledge (55.15 on MMLU, 28.18 on GPQA), consistent with Microsoft’s own framing of it as a small, task-focused model rather than a general-knowledge engine.
- Developers can run it now through Hugging Face Transformers with
trust_remote_code=True, and Microsoft has said quantized versions (likely GGUF, given the listed library support) will follow the initial unquantized bfloat16 release. - This release is explicitly a preview: Microsoft says the final Windows-shipped version will pair a further-optimized Aion with a separate content moderation model, targeting general availability by the end of January 2027.
- The model covers eight languages (English, German, Spanish, Portuguese, Italian, French, Chinese, Japanese), though Microsoft notes multilingual data is only about 6% of training data and output quality varies by language.
What makes Aion 1.0 different from a typical open-weight LLM?
Most open-weight releases aim to be generalists, competitive on broad knowledge and reasoning benchmarks. Aion 1.0 Instruct is built for a narrower job: running fast, private, and cheap on consumer hardware as part of the Windows operating system. Microsoft’s own model card is upfront about this trade-off, stating the model “was not trained to excel at world knowledge” and pointing developers toward its intended use cases: document comprehension, content analysis, question answering, summarization, and rewriting.
That focus shows up in the architecture. At roughly 2.29 billion parameters with tied input/output embeddings, Aion is small enough to run on-device without constant cloud connectivity. It uses 28 layers, a hidden size of 2048, and grouped-query attention (24 query heads, 8 key/value heads) to cut down on memory bandwidth. The more notable design choice is its hybrid attention pattern: 21 layers use sliding-window attention with a 2048-token window, while only 7 layers use full attention, extended out to the full 32,768-token context via YaRN scaling on top of RoPE positional encoding. This hybrid approach is a common technique for keeping long-context inference affordable on memory-constrained hardware, since full attention at every layer scales expensively with sequence length.
How does Aion 1.0 perform on benchmarks?
Microsoft published evaluation results using EleutherAI’s lm-evaluation-harness (with GPT-4o as judge where an LLM judge was needed). The pattern across categories reinforces the “small and task-focused” positioning:
- Knowledge: MMLU 55.15, MMLU-Pro 33.35, MMMLU (multilingual) 45.51, GPQA 28.18
- Math: GSM8K 74.91, GSMPlus 49.56, MATH-500 48.00
- Instruction following: IFEval 83.90, IFBench 32.00, Multi-IF 64.70
- Translation: chrF++ 43.18
The instruction-following numbers stand out as the strongest category, which makes sense given the model’s intended role: following user commands for rewriting, summarizing, and OS-level interactions rather than answering trivia or graduate-level science questions (GPQA). The gap between GSM8K (74.91) and the harder MATH-500 (48.00) also tracks with what you’d expect from a 2.3B model: solid on everyday arithmetic reasoning, weaker on multi-step formal math.
Is Aion 1.0 Instruct safe and reliable to use?
Microsoft’s model card is candid about limitations here too. It states plainly that Aion Instruct “does not have the complex reasoning needed to understand all the nuances of safety, including attempts to induce unsafe behavior,” and flags the model as potentially susceptible to adversarial attacks (prompt injection style manipulation). The card references internal safety metrics (XPIA, DIC, and ECI) as areas targeted for improvement in future releases, without publishing the underlying scores in detail beyond a chart.
Microsoft’s plan for the production Windows version addresses this gap by pairing Aion with a separate, dedicated content moderation model, rather than relying on Aion’s own built-in safety training alone. That combination (small task model plus a moderation layer) is the architecture Microsoft says will ship in Windows by general availability.
- Name
- Maria Alvarez
- Date of birth
- SSN
- Diagnosis
- Balance due
- $412.80
The card also lists clear out-of-scope uses: generating harmful or explicit content, creating misinformation, and automated decision-making in high-risk contexts like medical or legal judgments. These are standard caveats for any instruction-tuned model, but worth noting given Aion’s intended deployment inside an operating system used by hundreds of millions of people.
How do you run Aion 1.0 Instruct locally right now?
The model is available on Hugging Face as microsoft/Aion-1.0-Instruct-2.3B, released under the MIT license, and works with the standard Transformers library starting from version 5.x. Because it ships with custom modeling code (configuration_aion_instruct.py and modeling_aion_instruct.py), you need to set trust_remote_code=True when loading it.
A minimal setup looks like this:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "microsoft/Aion-1.0-Instruct-2.3B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype="auto",
device_map="auto",
trust_remote_code=True,
)
messages = [
{"role": "user", "content": "Give me a short introduction to large language models."},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Weights are published in bfloat16 as model.safetensors. Microsoft says this unquantized release is the first step, with quantized versions to follow, and the model’s listed library tags (PyTorch, ONNX, GGUF) suggest those future builds will target lighter-weight local inference paths, useful for anyone wanting to run it on more modest hardware than a full bfloat16 load requires.
The model also supports structured tool calling. Tool schemas get passed through apply_chat_template(..., tools=tools) and injected into the system prompt, with the model emitting calls in a <|tool_call|>{"name": ..., "arguments": {...}}<|/tool_call|> format, and results returned via a <|tool_response|> wrapped user turn. That’s relevant for anyone building agentic or OS-integration style workflows on top of it now, ahead of the Windows-native version.
What should developers expect from the Windows rollout?
The Hugging Face release is explicitly framed as a preview. Microsoft states that “a further optimized version of Aion Instruct, coupled with Microsoft’s content moderation model, will become the Windows OS SLM by the end of January 2027.” The company describes the public release as a way to maximize transparency and collect feedback before that integration happens, and points to a companion GitHub repository, microsoft/Aion-Instruct-Preview-Sample, for sample code.
For developers, that timeline matters less than the fact that the model’s behavior, chat format, and tool-calling interface are available to test now. Anyone building on top of Windows’ future on-device AI stack can start validating prompts, context handling, and tool integrations against the same base model architecture well before it ships system-wide.
Frequently Asked Questions
What is the difference between Aion 1.0 Instruct and Phi Silica?
Phi Silica is the current on-device SLM used in Windows. Aion 1.0 Instruct is its successor, trained on an extension of the same data used for Phi Silica, with additional synthetic data for math, coding, reasoning, and general world knowledge. Microsoft positions Aion as the next generation that will eventually replace Phi Silica as the default Windows OS model.
How much VRAM or hardware does Aion 1.0 Instruct need to run?
Microsoft’s model card doesn’t publish specific VRAM figures. As a roughly 2.3B-parameter model released in bfloat16, it’s small enough to run on many consumer GPUs and some CPUs with sufficient RAM, especially compared to multi-billion-parameter frontier models. Microsoft says quantized versions are coming, which will lower hardware requirements further once released.
Can Aion 1.0 Instruct be used commercially?
Yes. The model is released under the MIT license, one of the most permissive open-source licenses available, which allows commercial use, modification, and redistribution.
Does Aion 1.0 Instruct support languages other than English?
It supports eight languages: English, German, Spanish, Portuguese, Italian, French, Chinese, and Japanese. Microsoft notes that multilingual data makes up only about 6% of the total training data, and output quality can vary across languages as a result.
When will Aion power Windows directly?
Microsoft states that a further optimized version of Aion Instruct, combined with a separate content moderation model, is targeted for general availability in Windows by the end of January 2027. The current Hugging Face release is a preview meant to gather developer feedback ahead of that rollout.

