How to Run Baidu's Unlimited OCR 2 Locally for Document Parsing
Baidu's Unlimited OCR 2 parses long documents locally with SR-SWA attention. Here's how installation, VRAM use, and real tests actually hold up.
What is Unlimited OCR 2?
Unlimited OCR 2 is an open-weight document parsing model from Baidu, published on Hugging Face under the baidu/Unlimited-OCR-2 repository. It takes images of documents, handwritten pages, invoices, tables, and scanned text, and converts them into structured output such as clean text, tables, headings, and LaTeX for formulas. The model runs locally on a consumer GPU, uses the transformers library, and ships with its own inference code (modeling_unlimitedocr.py) plus a prebuilt SGLang wheel for serving it as a local API.
TL;DR
- Unlimited OCR 2 uses a mechanism called SR-SWA that keeps a small fixed set of reference tokens instead of attending to an entire document at once, which keeps memory and speed roughly flat as page count grows.
- A hands-on test showed the model consuming about 8.3GB of VRAM in full precision, making it runnable on a single mid-range GPU.
- On long-document benchmarks, the model reportedly holds around 77 out of 100 on a hundred-page test where many competing models drop under 30, a sharp jump from the original Unlimited OCR, which scored near zero on the same test.
- It handled a handwritten physics page well, converting equations for special relativity, the photoelectric effect, and blackbody radiation into readable LaTeX.
- It struggles with mixed-language documents, breaking down on a page combining English, Urdu, and Indonesian text, and it appears tuned mainly for English and Chinese.
- On a historical newspaper from 1883, it read body text quickly but misread the year as 1893, mangled a price, and skipped a decorative masthead graphic, showing it’s weaker on ornamental fonts and old print.
- Installation is straightforward: install prerequisites, download the model weights from Hugging Face, and run an inference script locally or via an SGLang server.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
How does SR-SWA work?
SR-SWA stands for the model’s approach to sparse, reference-based attention over long sequences. Instead of processing every page of a document as one giant attention context (which is how many vision-language models choke on long documents), the model maintains a small, fixed set of reference points and only looks closely at the parts of the document relevant to the current step.
A useful way to picture it: imagine a student transcribing notes from a massive stack of pages. Without this trick, the student tries to hold every page in memory at once, rereading the entire stack for every new line written. That’s slow and the cost grows with every added page. With SR-SWA, the student keeps only the handful of spots that matter right now and ignores the rest, staying fast and calm even as the stack grows arbitrarily large. The tradeoff is that the model is deliberately not trying to hold everything in view simultaneously, it’s trading exhaustive attention for consistent speed.
That design is why the model’s benchmark scores separate so clearly from competitors as document length increases. On benchmarks with only dozens of pages per document, Unlimited OCR 2 leads by a modest margin over models like Qwen and Gemini variants. But on a benchmark built around hundred-page documents, most tested models collapse to under 30 out of 100, while Unlimited OCR 2 stays around 77. The gap widens specifically because the architecture was built to avoid the usual memory and attention blowup that happens with very long inputs.
How do you install and run it locally?
Running Unlimited OCR 2 locally follows a pattern familiar to anyone who has deployed open Hugging Face vision-language models:
- Install prerequisites. This includes
transformers, PyTorch, and supporting Python packages the model’s code depends on. - Download the model weights. The repository hosts two safetensors shards plus configuration and custom modeling files (
modeling_unlimitedocr.py,deepencoder.py,configuration_deepseek_v2.py), indicating the architecture borrows components from DeepSeek’s V2 modeling code alongside Baidu’s custom encoder and sparse attention kernels (sparse_gather_triton.py,dash_attention.py). - Run an inference script that loads a local image and feeds it to the model, producing structured output (often rendered to an HTML page alongside the source image for comparison).
- Optionally, start an SGLang server. The repo bundles a prebuilt SGLang wheel (
sglang-0.5.20.dev10), letting you stand up a local inference server rather than running one-off scripts, which is more practical for batch processing or integrating OCR into a pipeline.
In testing, the full model loaded onto a GPU and settled at roughly 8.3GB of VRAM in full precision, a figure confirmed by watching GPU memory climb in real time as weights loaded. That number puts the model within reach of most modern consumer and prosumer GPUs without needing quantization, though running it on a rented cloud GPU is also an option if local hardware is tight.
Is Unlimited OCR 2 worth running for document parsing?
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
For clean, single-language documents, it performs well. A handwritten page of physics equations was converted into LaTeX with special relativity, the photoelectric effect, and blackbody radiation formulas rendered correctly. A printed math problem with a worked solution was parsed in about 2 seconds, correctly separating the question, title, and each equation step, with only a minor cosmetic slip (a missing fraction bar) while the actual numbers stayed accurate. An invoice containing both free text and tabular financial data was converted cleanly into text and table format, missing only a single trailing line, likely due to context length limits.
The model also held up across more than 50 pages of plain text in one test run without errors, which lines up with the architecture’s whole premise: long documents shouldn’t degrade performance.
Where it breaks down is multilingual content and non-standard typography. A page mixing English, Urdu, and Indonesian text was handled fine for the English lines, then the model fell into a repetition loop when it hit Urdu script, suggesting the model isn’t built for broad multilingual OCR despite the multilingual tag on its Hugging Face listing. It appears tuned primarily for English and Chinese. Historical or decorative print is another weak spot: a scanned 1883 newspaper was read quickly but the model misdated it as 1893, misread a price, and skipped an ornamental masthead graphic entirely.
The practical takeaway is that this is a strong tool for parsing long, plain-language documents (contracts, reports, scanned books, printed math or physics notes) where speed and consistency across many pages matter more than handling exotic scripts or antique typography. For anything involving old newspapers, decorative fonts, or multiple non-Chinese/English languages, manual review is necessary.
How does it compare to the first Unlimited OCR?
The earlier version of the model was already competent at OCR tasks but was explicitly a “long horizon” tool rather than an “unlimited horizon” one, meaning performance degraded as documents got very long. On the hundred-page benchmark, the original model reportedly scored close to zero, essentially failing outright once documents crossed a certain length. Unlimited OCR 2’s jump to around 77 on that same test is the headline improvement: it’s not a modest refinement, it’s a fix to the core limitation that document length was previously the bottleneck. The SR-SWA attention mechanism is the architectural reason this became possible, keeping compute and memory from growing in lockstep with page count.
Frequently Asked Questions
What is SR-SWA in Unlimited OCR 2?
SR-SWA is the model’s attention mechanism that keeps a small, fixed set of reference points in memory rather than attending over an entire document at once. This keeps speed and memory usage nearly flat even as document length grows significantly.
How much VRAM does Unlimited OCR 2 need?
In testing, running the model locally in full precision consumed approximately 8.3GB of VRAM, which fits comfortably on a single mid-range consumer GPU.
Does Unlimited OCR 2 support multiple languages?
It performs best on English and Chinese. Testing on a document mixing English, Urdu, and Indonesian showed it handled English correctly but failed on the Urdu text, falling into a repetition loop, despite being tagged as multilingual on Hugging Face.
How does Unlimited OCR 2 perform on long documents compared to other models?
On a hundred-page benchmark, it scores around 77 out of 100, while many competing vision-language models drop below 30. The original Unlimited OCR scored near zero on the same benchmark, making this version’s long-document handling its main improvement.
Can Unlimited OCR 2 read handwriting and old documents accurately?
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
It read handwritten physics equations into LaTeX accurately in testing. However, it struggled with a decorative 1883 newspaper, misreading the publication year and a price, and skipping an ornamental graphic, indicating weaker performance on antique or stylized print.
