What Is dots3-note Preview? Xiaohongshu's Open Multimodal MoE Model
dots3-note preview is Xiaohongshu's open-weight 280B MoE model with 16B active params, 512K context, and text, image, video, and audio input.
What is dots3-note preview?
dots3-note preview is an open-weight multimodal model released by dots studio, the AI lab inside Xiaohongshu (RedNote), the Chinese social platform. It’s a Mixture-of-Experts (MoE) model with 280 billion total parameters and 16 billion active parameters per token, supporting up to 512K tokens of context. It takes text, image, video, and audio as input and produces text output. It’s positioned as the first and most lightweight entry in a planned “dots3” family of models, released under the Apache 2.0 license.
TL;DR
- dots3-note preview is a 280B-parameter MoE model with only 16B parameters active per forward pass, making inference cheaper than its total size suggests.
- The model handles four input modalities, text, image, video, and audio, while always producing text output, and video inputs carry their audio track along with them.
- Context length reaches 512K tokens, putting it in the range of long-document and long-video analysis use cases.
- It ships in both BF16 and FP8 precision, with FP8 recommended for practical deployment on a single 8-GPU node.
- The architecture combines a dense layer plus 45 MoE layers, 256 routed experts with 1 shared expert, and a mix of sparse and sliding-window attention.
- It’s licensed under Apache 2.0, has day-one support paths through vLLM, SGLang, and Transformers, and is available for free trial through OpenRouter.
- dots studio frames this as the lightest model in a broader dots3 family, meaning heavier, more capable siblings are likely to follow.
Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
Who makes dots3-note preview, and why does Xiaohongshu have an AI lab?
Xiaohongshu (also known internationally as RedNote) is a Chinese lifestyle and social commerce platform with a large user base built around photo and video sharing, product discovery, and search-like content browsing. That business generates enormous volumes of multimodal content, images, video, captions, comments, so it’s not surprising the company runs an internal AI research group.
dots studio is that group’s model-development arm. dots3-note preview is described as the first open-weight release in the “dots3” family, suggesting a longer-term roadmap of models at different capability and cost tiers rather than a single one-off release. The naming (“note”) and the framing as the “most lightweight member of the family” imply that larger, more expensive-to-run dots3 variants are planned or already in development.
How does the MoE architecture work?
Mixture-of-Experts models split their parameters into many smaller “expert” sub-networks and route each token to only a handful of them, rather than running every parameter for every token. This lets a model have a very large total parameter count (more knowledge capacity) while keeping the actual compute cost per token much closer to that of a smaller dense model.
For dots3-note preview, the numbers break down like this:
- 280B total parameters, but only 16B activated per token.
- 256 routed experts plus 1 shared expert, with the router selecting the top 8 experts for each token.
- 1 dense layer followed by 45 MoE layers.
- A hidden size of 5120, with FFN hidden sizes of 13824 for the dense layer and 1536 per expert.
- A vocabulary of 152K tokens.
There’s also a Multi-Token Prediction (MTP) component, a shared layer of 1.13B parameters used for speculative decoding, which can meaningfully speed up generation without changing output quality.
On the attention side, the model mixes two mechanisms in roughly a 1:3 ratio: 13 layers of DSA (a sparse attention variant using a top-2048 selection scheme) and 33 layers of sliding-window attention (SWA). This combination is a common strategy for controlling the cost of very long context windows, since full dense attention over 512K tokens would be computationally prohibitive at this scale.
What can dots3-note preview actually do?
The model card lists a broad set of target capabilities rather than a narrow specialty:
- General knowledge and instruction following.
- Mathematical and logical reasoning.
- Tool use and multi-step agent workflows.
- Interactive tasks requiring exploration, memory updates, and adaptation.
- Code generation and code-based problem solving.
- Understanding of images, documents, charts, audio, and video.
- Long-context information processing.
The multimodal side is handled by dedicated encoders bolted onto the language backbone: a MoE-based Vision Transformer (7B total parameters, 1.2B activated) for images and video frames, and a dense 800M-parameter audio encoder for sound. Because video inputs include their audio track when present, the model can, for example, describe a concert clip while also referencing what’s audible in it, rather than treating video as silent frames.
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
Output is text-only. There’s no image or audio generation here, this is a model built to understand and reason over multimodal input, not to create new media.
How does it compare to other open multimodal models?
The model card includes benchmark charts covering general reasoning and agent tasks, plus multimodal understanding, benchmarked presumably against other open and possibly closed competitors, though exact comparison models and scores aren’t detailed in the released documentation beyond the chart images themselves. What is clear from the architecture is that dots3-note preview sits in a similar design space to other large-scale open MoE releases: very large total parameter counts paired with small active-parameter footprints, long context support, and multimodal encoders attached to a shared reasoning backbone.
The Apache 2.0 license is notable. It permits commercial use, modification, and redistribution without the restrictions that come with some other open-weight licenses, which matters for teams evaluating whether they can build products on top of it.
Is dots3-note preview practical to run?
Running a 280B-parameter MoE model is not a laptop exercise. The model card explicitly recommends serving the FP8 checkpoint on a single 8-GPU node using either SGLang or vLLM, both of which now have native or in-progress support for the architecture. Deployment examples target 8x NVIDIA H100 setups with tensor parallelism and expert parallelism both set to 8, using FP8 quantization and DeepGEMM or DeepEP backends to manage the MoE routing efficiently.
For teams without that kind of hardware, the FP8 checkpoint reduces memory requirements compared to BF16, and speculative decoding via the MTP/NEXTN mechanism can cut time-per-output-token by more than half, according to the model’s deployment notes. There’s also a “language-only” load option in both SGLang and vLLM for teams that only need the text reasoning capability and want to skip loading the vision and audio encoders entirely, saving further memory.
For quick evaluation without any local infrastructure, dots studio also lists a free tier of the model on OpenRouter, letting developers test the model’s behavior before committing to self-hosting.
Is dots3-note preview worth using?
Whether it fits a given project depends on the use case. If you need long-context, multimodal reasoning (documents plus images plus audio plus video in one conversation) and you’re comfortable running open-weight models at real infrastructure scale, it’s a legitimate option, especially given the permissive Apache 2.0 license and its native support in mainstream serving frameworks like vLLM and SGLang. If you need something you can run on a single consumer GPU, the total footprint (even with FP8 and MoE sparsity) puts it out of reach.
Its status as a “preview” and the “lightest” model in a planned family is also worth weighing. Early releases in a model family sometimes get superseded quickly by better-tuned or larger siblings, so teams building long-term dependencies should watch for updates from dots studio rather than treating this as a finished product.
Frequently Asked Questions
What does “MoE” mean for dots3-note preview specifically?
It means the model has 280 billion total parameters but only activates 16 billion of them for any given token, using a router that picks 8 out of 256 available experts (plus 1 always-on shared expert) per token. This keeps inference cost closer to that of a much smaller dense model while retaining a large total knowledge capacity.
What input types does dots3-note preview support?
Text, images, video, and audio. Video inputs include their audio track when available, so the model can reason about both visual content and sound in the same clip. Output is text only.
How much context can it handle?
Up to 512K tokens, which is enough for long documents, extended conversations, or lengthy video transcripts combined with other inputs in a single request.
What hardware do you need to run it?
The model card recommends an 8-GPU node (examples use 8x NVIDIA H100) running the FP8 checkpoint through vLLM or SGLang. BF16 requires even more memory. It is not designed for single-GPU or consumer hardware deployment.
Is dots3-note preview free to use?
The weights are released under the Apache 2.0 license, so they can be downloaded and self-hosted without licensing fees. dots studio also offers a free tier through OpenRouter for testing without local deployment.
