North MicroVision 2.4B: Installing Cohere's Local OCR Vision Model
Cohere's North MicroVision is a 2.4B open vision model for OCR and documents. Here's how to install it, its real VRAM use, and where it falls short.
What is North MicroVision 2.4B?
North MicroVision is a 2.4 billion parameter open vision-language model from Cohere Labs, released under the Apache 2.0 license and published on Hugging Face as CohereLabs/North-Micro-Vision-Instruct. It’s built for reading: OCR, forms, tables, charts, and documents, processed at native resolution rather than downscaled to a fixed size. It’s small enough to run on a single consumer or workstation GPU and, according to Cohere Labs, small enough to fine-tune on custom data.
TL;DR
- North MicroVision is a 2.4B parameter, Apache 2.0 licensed vision-language model from Cohere Labs built specifically for OCR, forms, tables, and document reading.
- The model uses native-resolution image processing, meaning it encodes documents and screenshots at their real size instead of resizing them down first, which matters for reading small text and dense tables.
- Its architecture runs a three-stage pipeline: a native-resolution vision encoder, a multimodal projector, and a 2 billion parameter language model that reads text and image tokens together.
- In hands-on testing on an RTX A6000, the fully loaded model consumed just over 5GB of VRAM, making it practical for most modern GPUs.
- Performance is uneven across languages: it handled clean printed English documents and structured invoices flawlessly, did well on cursive French, but struggled badly with Urdu and Arabic despite being marketed as multilingual.
- Installing it currently requires building transformers from source, not just a pip install, since support for the model architecture hasn’t landed in a stable release yet.
- The model’s clear sweet spot is printed, structured content: invoices, tables, and clean documents, rather than messy handwriting or non-European scripts.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
How does North MicroVision’s architecture work?
The model is organized as three stages that run in sequence. First, a native-resolution vision encoder ingests an image at whatever shape it comes in and converts it into visual tokens without cropping or shrinking away detail. Second, a multimodal projector maps those visual features into the same representational space the language model uses, effectively translating “image” into something the text model can read alongside words. Third, the North Micro LLM, a roughly 2 billion parameter language model, processes the combined text and image tokens and generates the output.
The language model itself uses a mixed attention pattern: several sliding window attention layers followed by one full attention layer, repeated across 28 layers total. This is a common technique in recent small language models to keep memory and compute costs down (sliding window attention only looks at a local chunk of context) while still preserving some full-context reasoning capability from the occasional full attention layer.
The model supports a wide set of languages according to its Hugging Face metadata, including English, German, French, Spanish, Italian, Portuguese, Hindi, Japanese, Korean, Chinese, and Arabic.
How do you install and run North MicroVision locally?
Setup follows a fairly standard local LLM workflow, but with one important catch. A demonstrated install on an Ubuntu machine with a single Nvidia RTX A6000 (48GB VRAM) went like this:
- Create an isolated Python environment (the demo used
uv). - Install PyTorch. Depending on your CUDA setup, you may need to skip pinning a specific version and just run
pip install torch. - Install
transformersfrom source, not from PyPI. As of this model’s release, the stable pip package doesn’t yet include support for its architecture, so a from-source install of the development branch is required for the model to load at all. - Download the model weights from Hugging Face using
huggingface-cli(orhf download). Unlike many Cohere models, North MicroVision-Instruct is not gated, so no access request is needed, though logging in with a Hugging Face token is still recommended to avoid rate limiting. - Run inference with a short Python script that loads an image, sets a text prompt, and asks the model to transcribe or describe it.
One practical note from testing: running the model against the dev branch of transformers throws some harmless warning noise in the terminal. These don’t affect output and can be ignored.
How much VRAM does North MicroVision actually need?
In testing, the fully loaded model consumed just over 5GB of VRAM. That’s a small footprint for a vision-language model and puts it comfortably within reach of consumer GPUs with 8GB or more, not just data-center cards. The test hardware (an RTX A6000 with 48GB) was massive overkill for this model specifically, and was mainly used because it was the machine on hand. For anyone deciding whether they can run this locally, 5GB is the number to plan around, with some headroom for the OS, other processes, and longer input images or prompts.
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
Is North MicroVision good at OCR and document reading?
It depends heavily on what kind of document you feed it. Across a series of tests:
Printed, structured documents: excellent. A printed invoice with a table, line items, addresses, and totals was extracted flawlessly, matching the source exactly. This is described as the model’s clear sweet spot: clean, printed, structured data.
Handwritten English: mixed. A handwritten multilingual note came back with several small errors, wrong capitalization, a hallucinated question mark, and a few misread words, even though the overall gist was captured.
Handwritten physics equations with LaTeX-style notation: partially strong. The model correctly transcribed several genuinely hard items, including the space-time interval, the Lorentz factor, and E = MC², but it lost accuracy on some symbols and fell into repetitive output toward the end of the harder equations.
Cursive French on a letterhead: strong. The model produced a near-perfect transcription and translation, and correctly separated the handwritten note from the printed letterhead and stamp, only slipping on a few flowing cursive words near the end.
Indonesian, Urdu, and Arabic: weak. Indonesian words were frequently misread (mixing up “want” and “may,” or “coffee” and “copy”). Urdu text wasn’t understood at all. A dense Arabic newspaper banner, meant to stress-test the multilingual claim, essentially fell apart.
The pattern that emerges: European languages and clean printed text perform well, consistent with what’s typically seen from Cohere’s language models generally. Non-European scripts and messy handwriting are where the 2.4B parameter budget shows its limits.
How does it compare to other compact vision models?
Cohere Labs’ own benchmarks pit North MicroVision against a lineup of other compact vision-language models across tasks like general visual question answering, multilingual understanding, and chart reading. According to those benchmarks, the model holds its own against rivals that are often larger, and it particularly stands out on document understanding and visual grounding tasks. That lines up with the hands-on results: it’s a document and OCR specialist first, and a general multilingual vision model second.
Is North MicroVision worth running locally?
For anyone doing OCR on printed documents, invoices, forms, or structured tables in English or major European languages, yes. The combination of a small footprint (around 5GB VRAM), an Apache 2.0 license with no usage restrictions, and strong performance on exactly the kind of documents most businesses deal with makes it a practical local tool. It’s also positioned as a good fine-tuning base given its size.
For multilingual OCR involving Arabic, Urdu, or other non-European scripts, or for reliably transcribing messy handwriting, it currently underperforms its own marketing claims and needs more validation before being trusted in production.
Frequently Asked Questions
What license is North MicroVision released under?
It’s released under Apache 2.0, a permissive open source license that allows commercial use, modification, and redistribution without royalty obligations.
How much VRAM does North MicroVision require?
In testing, the fully loaded model used just over 5GB of VRAM, making it runnable on most modern GPUs with 8GB of memory or more.
One coffee. One working app.
You bring the idea. Remy manages the project.
Do I need a Hugging Face account to download it?
An account and token are recommended to avoid download throttling, but the model itself is not gated, unlike some other Cohere releases, so no special access request is required.
Why does installation require building transformers from source?
Support for North MicroVision’s model architecture hadn’t yet been merged into the stable PyPI release of the transformers library at launch, so the development branch needs to be installed directly to run the model.
Which languages does North MicroVision handle well?
It performs strongly on English and major European languages (German, French, Spanish, Italian, Portuguese) as well as clean printed documents in general. It performs noticeably worse on Arabic, Urdu, and Indonesian, and on messy handwritten text in those languages.