Microsoft Aion 1.0 Instruct: A 2.3B On-Device Model Built for Windows
Microsoft's Aion 1.0 Instruct is a 2.3B parameter on-device model with 32K context, built to become the next Windows OS SLM.

What is Microsoft Aion 1.0 Instruct?
Aion 1.0 Instruct is a 2.3 billion parameter small language model built by Microsoft’s Windows Applied Sciences Group for on-device use. It’s designed to run directly on a PC rather than in the cloud, handling tasks like summarization, rewriting, and chat with a 32,768 token context window. Microsoft has published it on Hugging Face as an open-weight preview, with a more optimized version slated to become the built-in Windows OS model by the end of January 2027.
TL;DR
- Aion 1.0 Instruct is a 2.3B parameter instruction-tuned model released by Microsoft’s Windows Applied Sciences Group, meant to run locally on-device instead of calling out to the cloud.
- It supports a 32K token context window, enough to handle multi-page documents, long email threads, or technical specs without sending that data off the machine.
- Microsoft describes it as the successor to Phi Silica, the small language model currently powering on-device Windows AI features, and says Aion’s training data extends the Phi Silica dataset.
- The model card lists support for PyTorch, ONNX, and GGUF, signaling it’s meant to be deployable through standard local-inference toolchains, not just a research artifact.
- Microsoft is shipping the unquantized weights first, with quantized versions to follow, and plans to pair the model with a separate content moderation model when it ships in Windows.
- Benchmark numbers put it in typical small-model territory: 55.15 on MMLU, 74.91 on GSM8K, and 83.90 on IFEval, respectable for a 2.3B model but well behind frontier cloud models.
- The general availability target is end of January 2027, when a further-optimized version becomes the default Windows OS SLM.
Remy AI Engineer Certification
Your seat is open. Leave certified.
Build and deploy an app for enterprises with AI in one live afternoon. No coding.
How big is Aion 1.0 Instruct and what hardware does it need?
Aion 1.0 Instruct has roughly 2.29 billion parameters, with tied input and output embeddings to keep the footprint small. Architecturally, it’s a 28-layer transformer with a hidden size of 2048 and grouped-query attention (24 query heads, 8 key/value heads). It uses a hybrid attention pattern: 21 layers use sliding-window attention with a 2048-token window, and 7 layers use full attention extended out to the full 32K context via YaRN scaling on top of RoPE positional encoding.
That combination matters for anyone thinking about deployment. Sliding-window attention keeps compute and memory costs down for most of the network, while the handful of full-attention layers preserve the ability to reference information across the entire 32K window when needed. It’s a common pattern in recent small models built for constrained hardware, and it’s consistent with Microsoft’s stated goal of running this fully on consumer Windows devices rather than in a datacenter.
The model card doesn’t publish exact VRAM or RAM requirements, but the size class (2.3B parameters, bfloat16 weights) puts it in line with other on-device models that run comfortably on modern laptops with either a capable NPU, integrated GPU, or CPU fallback. The fact that Microsoft is releasing unquantized weights first, with quantized versions to follow, suggests the eventual Windows-shipped version will be considerably lighter than the Hugging Face release.
What formats and tooling does it support?
The model card lists three library targets: PyTorch, ONNX, and GGUF. That’s a notable spread. PyTorch support covers standard Hugging Face transformers workflows (the model card’s quickstart example uses AutoModelForCausalLM with trust_remote_code=True, since Aion uses custom modeling code). ONNX support points toward Windows’ own ONNX Runtime stack, which is how Microsoft typically ships on-device inference for its first-party AI features. GGUF support means the model is also expected to work with llama.cpp and the broader ecosystem of local-inference tools that have grown around that format.
Supporting all three suggests Microsoft wants both its own OS integration and third-party developers and tinkerers to be able to run the model locally, regardless of their preferred stack. The model also supports structured tool calling: tool schemas get injected into the system prompt, and the model emits calls in a <|tool_call|> format with JSON arguments, which matters for anyone building agentic workflows that need the model to invoke local functions rather than just generate text.
Is Aion 1.0 Instruct the successor to Phi Silica?
Yes, functionally. The model card states that Aion’s training data is “an extension of the data used for training Phi Silica (the previous generation Windows OS SLM).” Phi Silica has been the small model underpinning on-device Windows Copilot features. Aion 1.0 Instruct is positioned to replace it, with Microsoft explicitly calling Aion “the foundational Small Language Model (SLM) in the Windows ecosystem.”
The training mix combines filtered public documents with synthetic data generated to teach math, coding, common-sense reasoning, and general world knowledge. Multilingual data makes up about 6% of the total training set, and the model supports eight languages: English, German, Spanish, Portuguese, Italian, French, Chinese, and Japanese. Microsoft notes output quality still varies by language, and closing that gap is an ongoing effort.
How good is it, really, compared to larger models?
On benchmarks, Aion 1.0 Instruct performs like what it is: a small, efficient model, not a frontier one. It scores 55.15 on MMLU and 33.35 on the harder MMLU-Pro, 28.18 on GPQA, and 74.91 on GSM8K for math reasoning (dropping to 48.00 on MATH-500, a tougher math benchmark). On instruction following, it does better relatively speaking, hitting 83.90 on IFEval, though it drops to 32.00 on the stricter IFBench.
Microsoft is direct about the tradeoffs in its own model card: small language models like Aion “can miss fine details, provide misstatements, or show inability to follow complex multi-step reasoning,” and they lack the capacity of larger models to hold onto broad factual knowledge or follow intricate chains of logic. The model card also flags that Aion wasn’t trained to excel at world knowledge, and that it may be more susceptible to adversarial prompts designed to induce unsafe behavior, since it lacks the reasoning depth to reliably detect manipulation. Microsoft lists cross-prompt injection attacks (XPIA) and related safety metrics as areas targeted for improvement in future releases.
This is the expected tradeoff for an on-device model: you give up raw capability and world knowledge in exchange for speed, privacy, and the ability to run without network access. For tasks like summarizing a document that’s already open on your screen, drafting a quick rewrite, or handling a scoped chat interaction, that tradeoff is usually the point.
Why does an on-device model matter for Windows users and developers?
Running inference locally means the data never has to leave the device. For a 32K-token context window, that’s meaningful: full email threads, multi-page contracts, or technical specifications can be processed without being uploaded anywhere. It also means lower latency (no round trip to a server) and no dependency on connectivity or API costs for lightweight tasks.
For developers, Microsoft’s choice to publish this on Hugging Face under an MIT license, before it’s baked into Windows, is also notable. It’s an unusually transparent rollout for an OS-level AI component: rather than shipping the final model silently inside a Windows update, Microsoft is inviting feedback on the preview version now, with GA still more than a year out.
Frequently Asked Questions
What is Microsoft Aion 1.0 Instruct used for?
It’s built for general-purpose, on-device text tasks: document comprehension, summarization, rewriting, question answering, and chat. Microsoft explicitly says it’s not designed for high-risk uses like medical or legal decision-making.
How does Aion 1.0 Instruct compare to Phi Silica?
Aion is positioned as the successor to Phi Silica, the small model currently powering on-device Windows AI features. Its training data extends the Phi Silica dataset, and it’s set to replace Phi Silica as the default Windows OS SLM once the optimized, content-moderation-paired version ships.
When will Aion 1.0 Instruct be available in Windows?
Build an enterprise app with Dmitry Shapiro. Leave certified.
Microsoft states that a further optimized version, combined with a separate content moderation model, will become the Windows OS SLM by the end of January 2027. The current Hugging Face release is described as a preview meant to gather feedback ahead of that.
What formats does Aion 1.0 Instruct support for local deployment?
The model card lists PyTorch, ONNX, and GGUF support, covering standard Hugging Face workflows, Windows’ ONNX Runtime stack, and llama.cpp-style local inference tools. Microsoft is releasing unquantized weights first, with quantized versions to follow.
How large is Aion 1.0 Instruct and what’s its context window?
It has about 2.29 billion parameters and a context window of 32,768 tokens, enabling it to process long documents and multi-turn conversations entirely on-device.

