How to Run EVE Preview 4.5B Locally for Document Retrieval
A hands-on guide to installing Tencent's EVE Preview 4.5B visual retriever, testing it on invoices and citations, and building a full RAG pipeline.
What is EVE Preview 4.5B and why does it matter?
EVE Preview 4.5B is a visual document retriever released by Tencent that searches document pages as images instead of extracting text with OCR first. It’s a 4.5 billion parameter model that encodes both a text query and a document page image into sets of small vectors, then scores how well they match using a technique called late interaction. Instead of converting a PDF page into flat text and hoping the layout, tables, and charts survive the conversion, EVE looks directly at the pixels and finds which page answers your question. It reportedly ranks first on ViDoRe V3 and the earlier ViDoRe V1/V2 benchmarks, beating larger models by a wide margin.
The problem it targets is a familiar one for anyone who has built retrieval systems over scanned forms, financial filings, or technical reports. OCR is the weak link. Tables lose their row and column structure, charts turn into meaningless text fragments, and visual hierarchy (headers, emphasis, layout) disappears. EVE skips that step entirely by treating the page as an image from the start.
TL;DR
- EVE Preview 4.5B is a Tencent visual retriever that matches text queries against document page images directly, without any OCR preprocessing step.
- It uses token-level late interaction, the same scoring idea ColBERT introduced for text search, applied to visual document patches through a shared backbone.
- The model produces compact 128-dimensional vectors per token or patch, which keeps index sizes small, around 180 GB for a million pages according to the creator’s testing.
- In hands-on tests, the model correctly scored an invoice image higher for questions the invoice actually answered, without extracting the literal numbers itself.
- On a dense academic references page, EVE correctly matched queries about specific citations buried among a dozen others, showing it can localize fine detail inside a cluttered image.
- Loading the model onto a GPU used roughly 10 GB of VRAM in bfloat16, making it practical for a single consumer or workstation GPU.
- EVE is a retriever, not a reader: it finds the right page, but you still need a vision language model (like a reader VLM) to extract the actual answer text.
How does EVE’s architecture actually work?
The core idea is that EVE uses one shared backbone, not two separate encoders, for both the query and the document image. The transcript describes this backbone as a ColBERT 3.5 based network that processes text queries on one side and raw document page images on the other, using the same weights both times.
Each query word becomes its own 128-dimensional vector, and each visual patch of the document page becomes its own 128-dimensional vector. Critically, the page is never compressed into a single summary embedding. It stays a collection of vectors, each representing a small region of the page. This is what late interaction means in practice: instead of comparing one query vector to one document vector, the system compares every query token vector against every document patch vector and takes the maximum similarity (max-sim scoring) to produce a relevance score.
This approach traces back to ColBERT, which introduced the “keep all token vectors, score late” idea for text retrieval, and ColPali (referenced in the transcript as Pali), which first applied that idea to document images using a vision language backbone. EVE follows that same recipe (nicknamed ColBERT-Pali in the source material) but swaps in a stronger backbone and compresses vectors down to a native 128 dimensions, which is what keeps the index size manageable even at scale.
What do you need to run EVE Preview 4.5B locally?
Based on the hands-on setup, running EVE locally requires:
- A Linux system (Ubuntu was used in testing) with an NVIDIA GPU. The demonstration used an RTX A6000, though the model’s actual VRAM footprint during loading was closer to 10 GB in bfloat16 precision, suggesting smaller GPUs can likely handle it.
- A Python environment manager like
uvto create an isolated environment. - The vLLM inference engine, installed alongside the model’s other prerequisites, to serve the model efficiently.
- The model weights themselves, downloaded from Hugging Face on first run.
The practical installation flow is straightforward: create a virtual environment, install vLLM and the required dependencies, then run a script that loads the model in bfloat16, enables bidirectional attention (which the model needs for encoding), and points it at your document images and queries.
How do you test EVE on real documents?
A basic test script loads the EVE model onto the GPU, opens a document page as an image, encodes both the image and one or more text queries into token-level vectors, and runs max-sim scoring to produce a relevance score per query.
In one test, an invoice image was paired with two questions: “What is the total amount due?” and “What is the price of parts triple A?” Both queries scored high (15.19 and 14.12 respectively), correctly reflecting that the invoice image contained answers to both. Notably, the model did not output the actual dollar amounts. It only confirmed that the page was a strong match for each question. That distinction matters: EVE finds the right page, it doesn’t extract the answer from it.
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
A tougher test used a dense academic references page packed with citations, with a query asking which specific paper introduced “page attention.” The model again returned high scores for both test questions, correctly identifying that the answers were present even though each one was a single citation buried among many others on a visually cluttered page. This is a reasonable proxy for how the model would perform on real-world dense documents like financial tables, multi-column reports, or scanned legal forms.
Is EVE Preview 4.5B worth adding to a retrieval pipeline?
For teams already fighting OCR quality issues, EVE addresses a real gap. Standard OCR-based retrieval pipelines struggle with tables, multi-column layouts, embedded charts, and non-Latin scripts. Because EVE operates purely on page images, it sidesteps those failure modes entirely. It also produces compact 128-dimensional vectors, which keeps storage and index costs reasonable even at large scale.
The tradeoff is that EVE is only half a pipeline. It’s a retriever, not an answer generator. Once it identifies which page most likely contains an answer, you still need a reader model, such as a vision language model, to actually read the page image and extract the specific number, name, or fact. The full pipeline described in testing looks like this: EVE retrieves the top matching page image, that image is passed to a reader VLM, and the reader extracts the precise answer. Together, retriever and reader form a document RAG system that works directly on images rather than pre-extracted text.
For anyone building search over scanned invoices, technical reports, or multilingual filings, this two-stage approach (visual retrieval, then visual reading) avoids the brittleness of a pure OCR pipeline while still producing a final, readable answer.
Frequently Asked Questions
What makes EVE different from a standard OCR-based retriever?
EVE skips text extraction entirely and encodes document pages as raw images, matching them against text queries at the token and patch level. This avoids the structural damage OCR causes to tables, charts, and layout-heavy documents.
Does EVE Preview 4.5B extract answers from documents?
No. EVE is a retriever. It scores how well a page matches a query and returns the best-matching page, but it does not pull out the literal answer text. That job belongs to a separate reader model, typically a vision language model.
How much VRAM does EVE Preview 4.5B need?
In testing with bfloat16 precision on an RTX A6000, the model used roughly 10 GB of VRAM while loading, suggesting it’s feasible on more modest GPUs as well.
What is late interaction and why does EVE use it?
Late interaction, introduced by ColBERT, keeps a full set of token-level vectors for both the query and the document instead of compressing each into one summary vector. Relevance is computed by comparing all query vectors against all document vectors and taking the maximum similarity per query token. This preserves fine-grained detail, which is why EVE can find a single buried citation on a dense page.
Can EVE handle multilingual or non-Latin script documents?
The model’s design, working directly from pixels rather than extracted text, is well suited to multilingual and visually complex documents where OCR often introduces errors, though specific multilingual benchmark results weren’t covered in the tested material.